L-BFGS

[tanmay2798]

I wanted to ask that why after using L-BFGS with pytorch in https://i-systems.github.io/tutorial/KSNVE/220525/01_PINN.html significantly reduces the loss value?

[praksharma]

The question is fundamental to optimisers. While, second order optimisers are known for faster convergence, this is not generally true in cases where the loss lanscape contains saddle points.