Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hessian being called separately using only_fgh!() #1039

Open
acoh64 opened this issue Apr 30, 2023 · 1 comment
Open

Hessian being called separately using only_fgh!() #1039

acoh64 opened this issue Apr 30, 2023 · 1 comment

Comments

@acoh64
Copy link

acoh64 commented Apr 30, 2023

Hi,

I defined a function that computes f, g!, and h! together, as shown in the documentation. However, when I use this with only_fgh! to run NewtonTrustRegion(), the optimization algorithm computes the hessian separately from f and g!. My f, g!, and h! all use the results from an expensive calculation so it would greatly speed up the optimization if all three were computed at the same time.

Here is a MWE where you can see this behavior:

function test_fg!h!(F,G,H,p)
    print("Computing: ")
    if !(H == nothing)
        print("hessian, ")
        H[1] = 12.0*p[1]^2
    end
    if !(G == nothing)
        print("gradient, ")
        G[1] = 4.0*p[1]^3
    end
    if !(F == nothing)
        println("function")
        return p[1]^4
    end
end
res = Optim.optimize(Optim.only_fgh!(test_fg!h!), [2.0], NewtonTrustRegion(), Optim.Options(iterations = 100, show_every=1, show_trace=true))

Is it possible to fix this?

Thanks!

@pkofod
Copy link
Member

pkofod commented May 10, 2023

Hm, yes I appreciate the comment, and I am aware of the issue. The problem is, that there are many different combinations of what is expensive when called with what etc. I'll think about an option to force it. The problem is that in many cases (not all, thing quadratic or when a model solution is the major computational expense), adding the hessian calculation does indeed bring a computational cost. However, in trust region methods, it is often the case you might reject a step, reduce the radius, and then solve the subproblem again. The hessian does not update here, but you cannot figure out if you are in this case without evaluating the objective. You can create a cache variable that you only update when the parameters change. I think there's a tutorial somewhere. But I'll consider it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants