How to Run a Model on a GPU with bentoml in k8s?

[2232729885]

below is my bentofile.yaml:

'service: "service.py:svc"
include:

• "*.py"
  python:
  index_url: https://pypi.tuna.tsinghua.edu.cn/simple
  requirements_txt: "requirements.txt"
  docker:
  python_version: "3.8"
  cuda_version: "11.7.1"'

after bentoml build and containerize, i start a pod in k8s. I can run cmd 'nvidia-smi' in pod, the result is:

but no process run on gpu? so what i should do ?

[jianshen92]

Did you include SUPPORTED_RESOURCES = ("nvidia.com/gpu",) in your Runners?

[2232729885]

below is my service.py, can you tell me how to include SUPPORTED_RESOURCES = ("nvidia.com/gpu",) ?
`import bentoml

summarization_runner = bentoml.picklable_model.get("summarization:latest").to_runner()
svc = bentoml.Service("summarization",runners=[summarization_runner])

@svc.api(input=bentoml.io.JSON(), output=bentoml.io.JSON())
async def summarization(input_json):
return await summarization_runner.async_run(input_json)
`