You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Need support for loading models that only contain .pt weights
Motivation
I quantized Mixtral 8x7b model using HQQ (which produces a qmodel.pt file). But I am unable to load the weights in LoRAX as it expects either a .safetensors or .bin weights.
Your contribution
I haven't studied the source enough to submit a PR but from cursory understanding of the code, changes need to be made in hub.py file, specifically:
Feature request
Need support for loading models that only contain
.pt
weightsMotivation
I quantized Mixtral 8x7b model using HQQ (which produces a
qmodel.pt
file). But I am unable to load the weights in LoRAX as it expects either a.safetensors
or.bin
weights.Your contribution
I haven't studied the source enough to submit a PR but from cursory understanding of the code, changes need to be made in hub.py file, specifically:
lorax/server/lorax_server/utils/sources/hub.py
Lines 68 to 78 in cc2e0a9
Though I would also like to be able to load the base model from local rather than remote/from the hub (as explained in this issue: #347)
The text was updated successfully, but these errors were encountered: