Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to use the model that has already been downloaded? #127

Open
AntonioZC666 opened this issue Jan 7, 2024 · 1 comment
Open

How to use the model that has already been downloaded? #127

AntonioZC666 opened this issue Jan 7, 2024 · 1 comment

Comments

@AntonioZC666
Copy link

Hello, I'm trying to use FlexGen. I would like to know if I can run the project with a model that I have already downloaded? I'm using the case command line
python3 -m flexgen.flex_opt --model facebook/opt-1.3b
and it automatically downloads the model from Huggingface, but I've already downloaded it myself in advance. I tried using other parameter such as '--path' and adding the file path where my model is located, but it didn't work.

I just want to use a model that I've already downloaded, how should I do that?

@AntonioZC666
Copy link
Author

AntonioZC666 commented Jan 10, 2024

And I searched for the relevant Q&A and I found an argument --local from PR #111
I used a command python3 -m flexgen.flex_opt --model /mnt/data/share/llm/models/hf/opt-1.3b --local, but it didn't work. The error is flex_opt.py: error: unrecognized arguments: --local.
How can I make it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant