Add new Broyden version:

[JohannesNaegele]

In my use case the approximate norm descents slows down the computation by factor 10. Moreover, I can just initialize the inverse Jacobian with it's true value at the initial x without noticable additional costs.

I, however, don't know whether this is the proper approach to modify the API...

[ChrisRackauckas]

Interesting, what's the timing vs the Broydens in NonlinearSolve.jl? https://docs.sciml.ai/NonlinearSolve/stable/solvers/NonlinearSystemSolvers/

[JohannesNaegele]

Interesting, what's the timing vs the Broydens in NonlinearSolve.jl? https://docs.sciml.ai/NonlinearSolve/stable/solvers/NonlinearSystemSolvers/

Good point, I should probably just switch. TrustRegion() is about as fast as my NLsolve-modification (0.5s), however GeneralBroyden() is quite slow (3.5s). For LimitedMemoryBroyden() I get a DomainError; the solvers from SimpleNonlinearSolve apparently support only out-of-place functions (which took me quite some time to figure out lol).

[ChrisRackauckas]

@avik-pal take a look?

[avik-pal]

For LimitedMemoryBroyden() I get a DomainError

LimitedMemoryBroyden diverging is not surprising.

Moreover, I can just initialize the inverse Jacobian with it's true value at the initial x without noticable additional costs.

We can support a version of Broyden that initializes and resets with the true jacobian, without much difficulties. And I feel that can be the default for small problems that are diffentiable.

he solvers from SimpleNonlinearSolve apparently support only out-of-place functions (which took me quite some time to figure out lol).

SimpleNonlinearSolve supporting in place problems will land in the next few weeks (hopefully)

however GeneralBroyden() is quite slow (3.5s).

Is the GeneralBroyden problem, that it is not converging and hence taking 3.5s? (you can check with sol.retcode)

If you have the test problem somewhere public, I can take a look

[JohannesNaegele]

We can support a version of Broyden that initializes and resets with the true jacobian, without much difficulties. And I feel that can be the default for small problems that are diffentiable.

That would be great!

however GeneralBroyden() is quite slow (3.5s).

Is the GeneralBroyden problem, that it is not converging and hence taking 3.5s? (you can check with sol.retcode)

If you have the test problem somewhere public, I can take a look

I was unprecise in the sense that the reported times reflect 350 recursive solutions of a NL system. But nevertheless your guess is correct: Many of these solutions show no convergence. When I set maxiters to 350 (magic number copied from R package that implements the same economic models) the time reduces to 1.5s which is nevertheless still much slower than trust region.
https://github.com/JohannesNaegele/Consistent.jl/blob/1ffc0e966d8f0b9726c0f9f74b9785c2e1d69014/src/models/GROWTH.jl

[avik-pal]

Awesome, I will take a look at it!