Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Think about potential memory issues with refreshing and many gvars #20

Open
ckoerber opened this issue Oct 19, 2021 · 1 comment
Open
Labels
Feedback needed Please add a comment if you have a relevant idea or opinion help wanted Extra attention is needed

Comments

@ckoerber
Copy link
Owner

ckoerber commented Oct 19, 2021

To store correlations, gvars may accumulate over time when refreshing the app or refitting. If the data / priors are particularly large, this may cause memory issues.

We should try to delete caches each time a new fit is executed (and store the initial_fit / generate_fit result) but delete "temporary objects". As far as I can see, starting "fresh" is intended for most of the fits.

I believe, this should be relatively easy to implement with the current architecture but providing a setter method to FitGUI.fit.

@millernb: do you have any suggestions for how to best remove unneeded gvars?

@ckoerber ckoerber added this to To do in Public version Oct 20, 2021
@ckoerber ckoerber added Feedback needed Please add a comment if you have a relevant idea or opinion help wanted Extra attention is needed labels Oct 20, 2021
@ckoerber
Copy link
Owner Author

ckoerber commented Nov 8, 2021

Peter addressed a related issue: gplepage/gvar#3 (comment)

As far as I can tell, the "problematic action" is calling process_priors on several iterations (and computing different gvars based on the new prior).

It is generally possible to rebuild a smaller set of needed gvars inside or before/after process_priors using, e.g., gvar.rebuild. However, this might potentially crash due to globally defined gvars which are still used at another point (i.e., ones specified in additional_plots or so).

In short, it seems a bit more complicated than expected and may require having some sort of register of gvars which are used.

It might be possible to simplify this procedure if it is possible to explicitly remove the prior and derived quantities.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feedback needed Please add a comment if you have a relevant idea or opinion help wanted Extra attention is needed
Projects
Development

No branches or pull requests

1 participant