-
Notifications
You must be signed in to change notification settings - Fork 21.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
[optim] add missing 'maximize' parameter to LBFGS, NAdam and RAdam optimizers #126642
Comments
Yes, it is known that these do not support the flag, and the documentation for each exclude mentioning maximize for that reason. It is totally easily actionable to add maximize for RAdam and NAdam + tests, perhaps easier than adding the docs lol. I may just add this for funsies later this week but if someone wants to do this, go for it! LBFGS I've gotta check, but off the top of my head it should work as well, but the implementation is not immediately doable in my head. |
Fixes #[126642](#126642) I reference the maximize in `Adam` and add `Radam's` maximize flag. If this pr is OK, I will add another pr for `Nadam`. Pull Request resolved: #126765 Approved by: https://github.com/janeyx99
Fixes #[126642](pytorch#126642) I reference the maximize in `Adam` and add `Radam's` maximize flag. If this pr is OK, I will add another pr for `Nadam`. Pull Request resolved: pytorch#126765 Approved by: https://github.com/janeyx99
Fixes #[126642](pytorch#126642) I reference the maximize in `Adam` and add `Radam's` maximize flag. If this pr is OK, I will add another pr for `Nadam`. Pull Request resolved: pytorch#126765 Approved by: https://github.com/janeyx99
Reopening to track LBFGS |
Fixes pytorch#126642 Pull Request resolved: pytorch#127214 Approved by: https://github.com/janeyx99
馃殌 The feature, motivation and pitch
Currently,
LBFGS
,NAdam
andRAdam
optimizers do not have themaximize
parameter in their constructors, which means they cannot be used with some loss functions (like R2Score).Alternatives
In the case
maximize
can't be implemented due to a constraint of the optimizer, it would be nice to make it clear thatmaximize
is intentionally not supported. Some possible suggestions:Additional context
No response
cc @vincentqb @jbschlosser @albanD @janeyx99 @crcrpar
The text was updated successfully, but these errors were encountered: