Yatai & k8s

[danield137]

We are planning on using bento to manage our models.

Currently, our process looks like:

1. Post-training, make a bento, and save it on S3 (or equivalent)
2. Using a separate service, pull the bento, build the image and push to a container registry
3. On request - deploy to a Kubernetes cluster.

I would expect a way to track a specific bento to a pod. There is also a special consideration here, as I might want to use the same model with different hardware specs (gpu/cpu, different env variables) - this can create many bento permutations of the same base model, which I think might be problematic.

Also, notice that building an image is also a problem, is it requires a special step (docker inside docker).

Providing a Service for step 2 (as a dockerfile or a helm chart) can really help!

[jackyzha0]

Hi @danield137 , we have a guide for installing Yatai to a K8s cluster on the documentation here https://docs.bentoml.org/en/latest/guides/helm.html

Let me know if that solves your issue

[danield137]

Hi @jackyzha0, thanks for the quick reply.
I was actually asking about utilizing Yatai to manage deployments to k8s, as it is a multistep process, and currently no way to link a bento to a deployed pod AFAIK

[yubozhao]

@danield137 This is a great suggestion. We are going to enhance the functionalities of YataiService in the upcoming versions to make these operations a lot easier, like the ability to run Docker build process in a remote YataiService.

I will keep you updated within this discussion and love to get your opinions once we have more materials (design doc/etc) ready.