Server is busy

[hatrexltd]

2023-10-14T19:53:47+0000 [ERROR] [api_server:474] Exception on /text2img [POST] (trace=14018d3cdfaed397ec9c7385ef3aa282,span=fd90260af03a0807,sampled=0,service.name=onediffusion-sdxl-service)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/dist-packages/bentoml/_internal/server/http_app.py", line 343, in api_func
output = await run_in_threadpool(api.func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/starlette/concurrency.py", line 35, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/to_thread.py", line 33, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 2106, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 833, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/bentoml/bento/src/generated_stable_diffusion_xl_service.py", line 74, in text2img
res = model_runner.run(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/bentoml/_internal/runner/runner.py", line 52, in run
return self.runner._runner_handle.run_method(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/bentoml/_internal/runner/runner_handle/remote.py", line 355, in run_method
anyio.from_thread.run(
File "/usr/local/lib/python3.11/dist-packages/anyio/from_thread.py", line 45, in run
return async_backend.run_async_from_thread(func, args, token=token)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 2121, in run_async_from_thread
return f.result()
^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/usr/local/lib/python3.11/dist-packages/bentoml/_internal/runner/runner_handle/remote.py", line 248, in async_run_method
raise ServiceUnavailable(body.decode()) from None
bentoml.exceptions.ServiceUnavailable: Service Busy

Do you know why on certain requests there is this error with SDXL?

[aarnphm]

Is there a request sending before hand?