New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature Request] Option to print progress updates #280
Comments
Nice idea, but I'm not sure it's feasible. At the end of the day fitting is a minimization problem, and we don't know how many iterations we need to find the nearest local minimum (or even the value of the objective function at that point). |
Well, passing Even if the algorithm can't estimate how many iterations it has to go, even printing out how many iterations it's done so far, average time per iteration, and perhaps the the change in objective value for each iteration. I'm imagining something akin to the convergence plots in COMSOL, where you can see if it's at least heading in the right direction and how quickly. |
I definitely see the value of this. Implementing this is non-trivial though: Minimizers can call Objectives multiple times per iteration/step, e.g. to evaluate gradients. Which means we can't just patch something on According to [1] passing [1] https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html |
The callback is definitely the easiest way to do this since at least it is called only once per iteration, but it might also be scipy specific since other backends might not provide that. Then again, we can cross that bridge when we need to. I agree that changing the objective will probably require a lot of hacking to work. So I'm definitely in favor of leveraging a callback. Perhaps an API such as: from symfit.callbacks import FitStatus
fit = Fit(...)
fit_result = fit.execute(callback=FitStatus) where |
Sounds reasonable as a first attempt. The downside is that the callback will only be called with the parameter vector, so FitStatus would also have to know about the Objective. We can wrap this in a sort-of sane way in Fit with a verbose=True flag I think. |
It will also get the |
Only with trust-constr.
|
Why do I continue to get upset about these kinds of inconsistencies? Oh well, I guess we will indeed also have to provide the objective to The extra evaluation is kind of shit though, since this could impact performance quite a bit. We might have to include a warning to users about this. But I'm also thinking it might be better to decorate the objective in such a way as to add a cache of the last result, so we can retrieve that instead. If we store the current parameter vector + results in this cache, we can simply return the cache if the parameter vector matches the cached one and recalculate otherwise. |
Not a terrible idea. The downside is though, that caching will also invoke a performance penalty, which will then hit you always, even if you're running non-verbose. Maybe we could have Fit wrap it's objective in a cache if needed, but that's a strange road to go. I'll play around if I have some time. |
I agree with all of the above. Depending on the minimizer this caching might indeed be useless. But I think decorating the objective only when The floating error could be a problem but in principle shouldn't be so long as in scipy they use the exact same array to make both calls. |
It would be great to have a 'verbose' option to report the progress of the fit, either through a progress bar a la tqdm or just by printing out the current iteration. I've been trying some very ODE parameter estimations, and it would be nice to have an estimate of how long is left.
The text was updated successfully, but these errors were encountered: