Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Llama2 - LoRA Reference Implementation #727

Open
rgandikota opened this issue Apr 8, 2024 · 2 comments
Open

Llama2 - LoRA Reference Implementation #727

rgandikota opened this issue Apr 8, 2024 · 2 comments

Comments

@rgandikota
Copy link

rgandikota commented Apr 8, 2024

  1. The Readme points to using a eval.py, which is missing from the llama2 scripts folder.
  2. Instructions to run this reference implementation on multiple nodes would be helpful for anyone looking to use it as-is.
  3. Would be helpful to new submitters if the training time for the reference run is documented.
@rgandikota rgandikota changed the title Llama2 - LoRA missing eval.py script Llama2 - LoRA Submission Apr 8, 2024
@rgandikota rgandikota changed the title Llama2 - LoRA Submission Llama2 - LoRA Reference Implementation Apr 8, 2024
@itayhubara
Copy link
Contributor

  1. I'll delete it - it is just confusing
  2. @michal2409 can you open a PR to update the README with multi-node
  3. I can add one log but we don't usually do that - @nv-rborkar what do you think

@rgandikota
Copy link
Author

@itayhubara Could you please let us know if we can use this dataset from hugging face instead of the parquet files from the Google Drive? From the ReadMe instructions, looks like this is the dataset the parquet files
https://huggingface.co/datasets/tau/scrolls/blob/main/gov_report.zip

@rgandikota rgandikota reopened this May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants