bentoml-transformers-tutorial

1. Installation

   • BentoML requires Python 3.7 or above.
   • pip install -r requirements.txt
   • Install torch for your version

2. Creating a Service

   1. python3 bento_packer.py

   2. bentoml list

   3. run : bentoml serve TransformerService:latest

   4. Go to http://localhost:5000 and check the connection

   5. Send prediction request to the service:

   6. Using Python3

      import requests
      res = requests.post("http://127.0.0.1:5000/predict", json={"text": "transformer"})
      print(res.text)
      
      ## "<pad> 트랜스포머</s>"
      

3. Running the Yatai server : bentoml yatai-service-start

   Go to http://localhost:3000 and check the connection

4. Generate Docker Image

   • bentoml containerize TransformerService:latest
   • For Mac With Apple Silicon bentoml containerize --platform=linux/amd64 iris_classifier:latest

Running load tests using Locust

Run : locust -f locust.py

Go to http://localhost:8089 and check the connection

• Number of users : maximum number of users
• Spawn rate : User creations per second
• Host : Target address

Reference

• https://zuminternet.github.io/BentoML/
• https://velog.io/@perfitt/BentoML-%EC%82%BD%EC%A7%88%EA%B8%B0
• https://velog.io/@gtpgg1013/MLOps-BentoML-1
• https://etloveguitar.tistory.com/145