You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Oct 3, 2023. It is now read-only.
In the case of downscaling the servers, we don't give any grace time to process the backlogs it might have.
This results in the loss of some client requests.
A better way would be to soft delete the server first, wait for a minute, and finally, stop the LightningWork.
Soft delete: just remove the server from the LoadBalancer.servers list but don't stop the ModelServing work.
In the case of downscaling the servers, we don't give any grace time to process the backlogs it might have.
This results in the loss of some client requests.
A better way would be to soft delete the server first, wait for a minute, and finally, stop the LightningWork.
Soft delete: just remove the server from the
LoadBalancer.servers
list but don't stop theModelServing
work.Here is how Kubernetes do it.
IMO, the best way to do this would be overriding the on_exit method.
The text was updated successfully, but these errors were encountered: