-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: assert parts[0] == "base_model" AssertionError #4883
Comments
Traceback (most recent call last): The above exception was the direct cause of the following exception: Traceback (most recent call last): The above exception was the direct cause of the following exception: Traceback (most recent call last): The above exception was the direct cause of the following exception: Traceback (most recent call last): |
#3177 |
Your current environment
🐛 Describe the bug
CUDA_VISIBLE_DEVICES=0,1 python -m vllm.entrypoints.openai.api_server
--model /home/greatwall/app/edison/models/Qwen1.5-14B-Chat
--trust-remote-code
--served-model-name qwen14B
--max-model-len 4096
--gpu-memory-utilization 0.9
--enable-lora
--lora-modules lora1=/home/greatwall/app/edison/output/qwen1half-14b-chat/v65-20240515-143141/checkpoint-1110
--host 0.0.0.0
--port 8088
--tensor-parallel-size 2
--enforce-eager
The text was updated successfully, but these errors were encountered: