You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Then when we get an error when loading lora checkpoint (e.g. lora rank > max_lora_rank), all processing requests would fail (no matter whether other requests use lora).
The text was updated successfully, but these errors were encountered:
Your current environment
馃悰 Describe the bug
Vllm load lora checkpoints when executing model
https://github.com/vllm-project/vllm/blob/v0.4.2/vllm/worker/model_runner.py#L789-L790
https://github.com/vllm-project/vllm/blob/v0.4.2/vllm/lora/worker_manager.py#L138-L172
Then when we get an error when loading lora checkpoint (e.g. lora rank > max_lora_rank), all processing requests would fail (no matter whether other requests use lora).
The text was updated successfully, but these errors were encountered: