Add Halley's method via descent API #404
Conversation
src/descent/halley.jl
Outdated
| end | ||
| b = cache.b | ||
| # compute the hessian-vector-vector product | ||
| hvvp = derivative(x -> cache.f(x, cache.p), u, δu, 2) |
There was a problem hiding this comment.
| hvvp = derivative(x -> cache.f(x, cache.p), u, δu, 2) | |
| hvvp = derivative(Base.Fix2(cache.f, cache.p), u, δu, 2) |
There was a problem hiding this comment.
This will also need a case for inplace cache.f
There was a problem hiding this comment.
Got it. TaylorDiff.jl haven't got a in-place mode yet, but will do this week
src/descent/halley.jl
Outdated
| Compute the descent direction as ``J δu = -fu``. For non-square Jacobian problems, this is | ||
| commonly referred to as the Gauss-Newton Descent. |
src/algorithms/halley.jl
Outdated
|
|
||
| An experimental Halley's method implementation. |
There was a problem hiding this comment.
Ig you will change this later. But keeping it marked so that it doesn't fall through.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #404 +/- ##
===========================================
- Coverage 86.46% 51.76% -34.71%
===========================================
Files 47 49 +2
Lines 2904 2954 +50
===========================================
- Hits 2511 1529 -982
- Misses 393 1425 +1032 ☔ View full report in Codecov by Sentry. |
|
@avik-pal Support for in-place I think there isn't an eqivalent for |
Yes mark it as |
test/misc/halley_tests.jl
Outdated
| @testitem "Halley method" begin | ||
| f(u, p) = u .* u .- p | ||
| f!(fu, u, p) = fu .= u .* u .- p | ||
| u0 = [1.0, 1.0] | ||
| p = 2.0 | ||
|
|
||
| # out-of-place | ||
| prob1 = NonlinearProblem(f, u0, p) | ||
| sol1 = solve(prob1, Halley()) | ||
| @test sol1.u ≈ [sqrt(2.0), sqrt(2.0)] | ||
|
|
||
| # in-place | ||
| prob2 = NonlinearProblem(f!, u0, p) | ||
| sol2 = solve(prob2, Halley()) | ||
| @test sol2.u ≈ [sqrt(2.0), sqrt(2.0)] | ||
| end |
There was a problem hiding this comment.
Let's add a bit more comprehensive tests here similar to SimpleNonlinearSolve
|
I am trying to add this as a block in
|
|
Yeah ReTestItems doesn't yet work with VSCode Test Runner. In a julia REPL activate the test using TestEnv then do using ReTestItems
ReTestItems.runtests(<path to file or test directory>; name="<name of the testitem>") |
|
Ok I added it to this file near |
|
@avik-pal Do you have an out-of-place version of 23 problems? When testing with |
|
Don't the tests in simplenonlinearsolve use the oop version? |
|
So currently, the tests at https://github.com/SciML/SimpleNonlinearSolve.jl/blob/main/test/core/23_test_problems_tests.jl doesn't include Halley's method, and when I tried to add Halley in the same way as Newton, I get the error above. And the file at https://github.com/SciML/DiffEqProblemLibrary.jl/blob/master/lib/NonlinearProblemLibrary/src/NonlinearProblemLibrary.jl doesn't have out-of-place |
|
Ok so I manually ran |
Add Halley's method by computing Hessian-vector-vector product with TaylorDiff.jl in$O(n)$ time. Note that this isn't structured as something like "HessianCache", but instead structured as a descent method, which can be viewed as "adding some correction term to the Newton's method". This is mainly because Hessian, either implicitly or explicitly, isn't likely to be used elsewhere.
Tests haven't been added because TaylorDiff.jl should be added to an extension. Once that is done, it is straightforward to add tests.
Checklist
contributor guidelines, in particular the SciML Style Guide and
COLPRAC.
Additional context
Add any other context about the problem here.