Replies: 1 comment
-
@AbhiDub Hi! Did you test Kserve 0.11? There are some improvements about logging, maybe you are missing some exceptions etc.. Are you running a custom isvc? If so, did you check if your code logs failure scenarios etc..? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
kfserv inference pipeline returns response code 500 for each request. How to resolve it ?
Defaulted container "kserve-container" out of: kserve-container, queue-proxy, storage-initializer (init) 2023-07-20 15:35:42.486 1 root INFO [download():63] Copying contents of /mnt/models to local 2023-07-20 15:35:42.506 1 root INFO [register_model():159] Registering model: lgbm-iris-v2 2023-07-20 15:35:42.507 1 root INFO [start():126] Setting max asyncio worker threads as 5 2023-07-20 15:35:42.508 1 root INFO [serve():136] starting uvicorn with 1 workers 2023-07-20 15:35:42.571 1 root INFO [start():62] Starting gRPC server on [::]:8081 2023-07-20 15:35:42.578 12 uvicorn.error INFO [serve():84] Started server process [12] 2023-07-20 15:35:42.579 12 uvicorn.error INFO [startup():45] Waiting for application startup. 2023-07-20 15:35:42 DEBUG [timing_asgi.middleware:40] ASGI scope of type lifespan is not supported yet 2023-07-20 15:35:42.579 12 uvicorn.error INFO [startup():59] Application startup complete. 2023-07-20 17:06:22.164 12 root INFO [timing():49] [kserve.io](http://kserve.io/).kserve.protocol.rest.v1_endpoints.predict 0.0017910003662109375, ['http_status:500', 'http_method:POST', 'time:wall'] 2023-07-20 17:06:22.164 12 root INFO [timing():49] [kserve.io](http://kserve.io/).kserve.protocol.rest.v1_endpoints.predict 0.0017580000000005924, ['http_status:500', 'http_method:POST', 'time:cpu'] 2023-07-20 17:07:39.253 12 root INFO [timing():49] [kserve.io](http://kserve.io/).kserve.protocol.rest.v1_endpoints.predict 0.006926298141479492, ['http_status:500', 'http_method:POST', 'time:wall']
Below is the inference service yaml
Name: lgbm-iris-v2 Namespace: kubeflow-user-example-com Labels: <none> Annotations: sidecar.istio.io/inject: false API Version: serving.kserve.io/v1beta1 Kind: InferenceService Metadata: Creation Timestamp: 2023-07-03T12:29:47Z Finalizers: inferenceservice.finalizers Generation: 1 Managed Fields: API Version: serving.kserve.io/v1beta1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:annotations: .: f:sidecar.istio.io/inject: f:spec: .: f:predictor: .: f:lightgbm: .: f:name: f:storageUri: Manager: OpenAPI-Generator Operation: Update Time: 2023-07-03T12:29:45Z API Version: serving.kserve.io/v1beta1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:finalizers: .: v:"inferenceservice.finalizers": Manager: manager Operation: Update Time: 2023-07-03T12:29:47Z API Version: serving.kserve.io/v1beta1 Fields Type: FieldsV1 fieldsV1: f:status: .: f:address: .: f:url: f:components: .: f:predictor: .: f:address: .: f:url: f:latestCreatedRevision: f:latestReadyRevision: f:latestRolledoutRevision: f:traffic: f:url: f:conditions: f:modelStatus: .: f:copies: .: f:failedCopies: f:totalCopies: f:states: .: f:activeModelState: f:targetModelState: f:transitionStatus: f:observedGeneration: f:url: Manager: manager Operation: Update Subresource: status Time: 2023-07-03T12:30:19Z Resource Version: 161969482 UID: c7e62b53-fabb-4c5b-badf-61363acca658 Spec: Predictor: Model: Model Format: Name: lightgbm Name: Resources: Storage Uri: s3://pl-kubeflow-application-bucket/kserve_serving Status: Address: URL: http://lgbm-iris-v2.kubeflow-user-example-com.svc.cluster.local/v1/models/lgbm-iris-v2:predict Components: Predictor: Address: URL: http://lgbm-iris-v2-predictor-default.kubeflow-user-example-com.svc.cluster.local/ Latest Created Revision: lgbm-iris-v2-predictor-default-00002 Latest Ready Revision: lgbm-iris-v2-predictor-default-00001 Latest Rolledout Revision: lgbm-iris-v2-predictor-default-00001 Traffic: Latest Revision: true Percent: 100 Revision Name: lgbm-iris-v2-predictor-default-00001 URL: http://lgbm-iris-v2-predictor-default.kubeflow-user-example-com.svc.cluster.local/ Conditions: Last Transition Time: 2023-07-20T22:06:20Z Reason: Predictor ingress not created Status: False Type: IngressReady Last Transition Time: 2023-07-20T22:06:20Z Severity: Info Status: Unknown Type: PredictorConfigurationReady Last Transition Time: 2023-07-20T22:06:20Z Status: Unknown Type: PredictorReady Last Transition Time: 2023-07-20T15:35:44Z Severity: Info Status: True Type: PredictorRouteReady Last Transition Time: 2023-07-20T22:06:20Z Reason: Predictor ingress not created Status: False Type: Ready Model Status: Copies: Failed Copies: 0 Total Copies: 1 States: Active Model State: Loaded Target Model State: Pending Transition Status: InProgress Observed Generation: 2 URL: http://lgbm-iris-v2.kubeflow-user-example-com.svc.cluster.local/ Events: <none>
Beta Was this translation helpful? Give feedback.
All reactions