Do maximize=True for dual_optimizers

[juan43ramirez]

Enhancement

Pytorch optimizers include a maximize flag (Pytorch Issue)*. When set to True, the sign of gradients is flipped inside optimizer.step() before computing parameter updates. This enables gradient ascent steps natively.

NOTE: This sign flip does not affect the parameter's .grad attribute.

Cooper currently populates the gradients of dual variables with their negative value, so that descent steps performed by the dual optimizer are in fact ascent steps towards optimizing the problem formulation.

cooper/cooper/constrained_optimizer.py

Lines 401 to 406 in 09df759

	# Flip gradients for multipliers to perform ascent.
	# We only do the flipping *right before* applying the optimizer step to
	# avoid accidental double sign flips.
	for multiplier in self.formulation.state():
	if multiplier is not None:
	multiplier.grad.mul_(-1.0)

We should not do the sign flipping manually but rather force set maximize=True when instantiating the dual optimizer.

* This has been implemented on on Pytorch's master branch for every optimizer but LBFGS . On v1.12, Adam, SGD and AdaGrad support the flag, but not RMSProp. An assert could be included to ensure that the requested dual optimizer supports the flag.

⚠ This change would break compatibility with versions of Pytorch prior to 1.12.

Motivation

Manually flipping gradients immediately after calculating them (thus ensuring that this happens before calls to dual_optimizer.step()) is error prone.
Moreover, keeping track of the fact that gradients have a sign flipped is inconvenient.

By implementing this change we would adopt the official Pytorch approach for performing ascent steps.

Alternatives

The current implementation is functional.

References

• Add the maximize flag to all optimizers pytorch/pytorch#68052

[juan43ramirez]

This would require changing the Extra optimizers to also have the flag