Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gradient checkpointing issue when running QLoRA finetuning #413

Open
tytung2020 opened this issue Jul 1, 2023 · 1 comment
Open

Gradient checkpointing issue when running QLoRA finetuning #413

tytung2020 opened this issue Jul 1, 2023 · 1 comment
Assignees
@codestar12 @danbider
Labels
question Further information is requested

Comments

@tytung2020
Copy link

tytung2020 commented Jul 1, 2023 •
edited

Finetuning the mpt-7b and mpt-30b using qlora gives the error "ValueError: MPTForCausalLM does not support gradient checkpointing.". Is there a way to fix this?
@tytung2020 tytung2020 added the question Further information is requested label Jul 1, 2023
@minionKP minionKP mentioned this issue Jul 3, 2023
Closed
@tytung2020
Copy link
Author

tytung2020 commented Jul 12, 2023 •
edited

are these lines of codes what is needed to make it work? cekal's amendment seems to work on the 7b version:
https://huggingface.co/cekal/mpt-7b-peft-compatible/commit/a5eab52c1c61c1d50a4e01428949f6ff90c73c48
But not sure if it works fully as intended. Could someone in MosaicML check this?
If so, please also implement this in the 30b version. Thanks~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@codestar12 codestar12

@danbider danbider

Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@codestar12 @tytung2020 @danbider