[Q] How to call predictor backend independently from trasformer #3330
-
I'm following this docs example on using triton as the predictor with a transformer. I tried a couple of things but I just can't figure out the right call address/port api signature. It always seems to route to the transformer. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
@cceyda Transformer and Predictor both have separate URLs. You can bypass the transformer by using the predictor URL. You can find it in the isvc status. |
Beta Was this translation helpful? Give feedback.
@cceyda Transformer and Predictor both have separate URLs. You can bypass the transformer by using the predictor URL. You can find it in the isvc status.