You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We saw non-deterministic model responses when doing greedy decoding using evaluate/predict.py, and disabling vllm fixed this issue. Two potential fixes: upcast models to float32, and upgrade vllm to a later version
The text was updated successfully, but these errors were encountered:
Conversation in slack: https://allenai.slack.com/archives/C06GS4HAWJV/p1712773278326539
Similar issue: vllm-project/vllm#966
We saw non-deterministic model responses when doing greedy decoding using evaluate/predict.py, and disabling vllm fixed this issue. Two potential fixes: upcast models to float32, and upgrade vllm to a later version
The text was updated successfully, but these errors were encountered: