New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix loading optimizer options from archive #125215
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/125215
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit ca188ac with merge base e0d2c24 (): This comment was automatically generated by Dr. CI and updates every 15 minutes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, can you add a test case
@janeyx99 I update the existing test for optimizer serializing. Now the test passes with the proposed patch but fails without it. |
test/cpp/api/serialize.cpp
Outdated
const double lr = param_group.options().get_lr(); | ||
// change the learning rate, which will be overwritten by the loading | ||
// otherwise, test cannot check if options are saved and loaded correctly | ||
param_group.options().set_lr(lr + 1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lrs are usually sub 1 haha, can we change this to + .01?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay. I change it to 0.01
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
This PR makes libtorch behave the same as PyTorch when loading optimizer state from archive. With PyTorch, options of parameter groups are loaded from the archive, which is missing currently in libtorch.