Ensemble/BLS models where individual steps are hosted on different machines (or clusters)? #7166
Unanswered
vadimkantorov
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Is it supported by Ensemble models or BLS models the scheme where individual pipeline component / model is scheduled on a different machine?
E.g. it might make sense to host/scale certain preprocessing component / instance group on a big CPU machine (or even some cluster) and run another step on a GPU machine.
I guess for this during BLS/Ensemble config we'd need to specify URL endpoint for individual steps
Is sth like this supported natively by the Triton Server?
Thanks!
Beta Was this translation helpful? Give feedback.
All reactions