Request for unknown model: 'test_model_pytorch' version 1 is not at ready state

[takumi5757]

https://github.com/allegroai/clearml-serving/tree/main/examples/pytorch

I'm running examples as per readme.md, but I get the following error.

What should I do?

{"detail":"Error processing request: <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"Request for unknown model: 'test_model_pytorch' version 1 is not at ready state\"\n\tdebug_error_string = \"{\"created\":\"@1652700912.192078289\",\"description\":\"Error received from peer ipv4:172.25.0.5:8001\",\"file\":\"src/core/lib/surface/call.cc\",\"file_line\":1069,\"grpc_message\":\"Request for unknown model: 'test_model_pytorch' version 1 is not at ready state\",\"grpc_status\":14}\"\n>"}

Endpoints appear to be normal.

[jkhenning]

Hi @takumi5757,

This seems like triton pytorch issue...

• What is the triton container you are using? is it the clearml-serving-triton container?
• What is the driver version of the host machine running triton? As far as I know, only the latest triton version works with pytorch, and for that to work you need Nvidia drivers 5xx
• Can you share the full log of the triton container?

[takumi5757]

Thank you @jkhenning.

Yes, it is the clearml-serving-triton container.
I updated the Nvidia driver from 490 to 515 and fixed it.

Thank you for your response.