Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The noisemodel keyword argument in ps.Model and noise keyword argument in ml.solve are Deprecated as of Pastas 1.5 #735

Open
raoulcollenteur opened this issue Apr 11, 2024 · 0 comments
Assignees
Labels
development Indicates development of new features priority 0 high, deal with as soon as possible
Milestone

Comments

@raoulcollenteur
Copy link
Member

raoulcollenteur commented Apr 11, 2024

This thread is used to explain users the change of deprecating the noisemodel in ps.Model and noise argument in ml.solve and ml.initialize methods. The error messages are linked to this issue, which is pinned at the top of the issue page.

Changes to default behavior regarding noise models in pastas>=1.5

Noise models are no longer added to time series models by default. From Pastas 1.5
onwards we are forcing users to explicitly add a noise model to their time series model
should they want to use one.

Warning

This means that if you are running old code with the new version of
Pastas and haven't specified anything with respect to the noise model in your code
(thereby relying on Pastas defaults) the result will change !

Contents

Remind me, what is a noise model?

A noise model can be used to try to reduce the residuals (simulation minus
observations) to white noise. When the noise (the result after applying the noise model
to the residuals) meets the criteria of white noise, this means we can make certain
statistical inferences with our time series model. For example, when the noise meets
these criteria, the estimated parameter uncertainty is reliable and can be used to
quantify model uncertainty. For more information about the noise model, check out the
following notebooks:

Why are we changing the default in Pastas?

There are several reasons:

  • A noise model is not always necessary in a time series model.
  • We want users to explicitly choose whether they want to use a noise model. This means an extra line of code is sometimes necessary, but this makes it much clearer what the modeler is doing.
  • There are other methods for estimating the uncertainty of the model parameters that
    do not require a noise model (e.g. Monte Carlo Markov Chain available with the Emcee solver).
  • Make it easier to to teach Pastas. Beginners are not confronted by the complexities
    of the noise model right away. Instead the complexity of the time series models can
    be increased progressively as users gain more insight into time series modeling.
  • Noise models can sometimes impact the model results and cause a bad model fit.

Overview of changes

Starting from pastas 1.5:

  • Models will not have a noise model added by default.
  • The noisemodel keyword argument in ps.Model() is removed.
  • The noise keyword argument in ml.solve() is removed.
  • The estimates of the standard errors of the parameters are not shown in the model fit
    report (ml.fit_report()) and the model results plot (ml.plots.results()). These
    can be shown by passing stderr=True to either function.
  • The noise models have been renamed to ps.ArNoiseModel (was ps.NoiseModel) and
    ps.ArmaNoiseModel (was ps.ArmaModel).

Guide for updating to Pastas 1.5

The following section explains how to update your code for pastas>=1.5.
Where applicable, the equivalent code for pastas<=1.4 and pastas>=1.5 is shown.

Adding a noise model

In pastas<=1.4 a noise model was added automatically, unless users opted-out.
In pastas>=1.5 a noise model has to be added explicitly:

ml = ps.Model(head)
ml.add_noisemodel(ps.ArNoiseModel())

Reproducing old default behavior (modeling with a noise model)

In pastas<=1.4, this was the default:

ml = ps.Model(head)
sm = ps.RechargeModel(prec, evap)
ml.add_stressmodel(sm)
ml.solve()

To reproduce this result in pastas>=1.5:

ml = ps.Model(head)
sm = ps.RechargeModel(prec, evap)
ml.add_stressmodel(sm)
ml.add_noisemodel(ps.ArNoiseModel())  # users have to explicitly add a noise model
ml.solve()

Solving a model without and then with a noise model (two-step solve)

In pastas<=1.4 this could be done as follows:

ml = ps.Model(head)
sm = ps.RechargeModel(prec, evap)
ml.add_stressmodel(sm)
ml.solve(noise=False)
ml.solve(noise=True, initial=False)

In pastas>=1.5, this now becomes:

ml = ps.Model(head)
sm = ps.RechargeModel(prec, evap)
ml.add_stressmodel(sm)
ml.solve()                            # first solve without a noise model
ml.add_noisemodel(ps.ArNoiseModel())  # add the noise model
ml.solve(initial=False)               # solve using initial=False to use the optimized parameters from the previous solve

Solving without a noise model

If a model already has a noise model (for example, when loading old stored models).

Pastas<=1.4:

ml.solve(noise=False)

Pastas>=1.5

ml.del_noisemodel()
ml.solve()

Displaying the estimated parameter uncertainty

In pastas<=1.4 the estimated parameter uncertainty was always shown in the model fit
report and model results plot. This was not always appropriate as these errors are only
reliable if the residuals meet certain criteria. The user now has to explicitly define
whether to show these estimates.

To show the parameter uncertainty estimates in the fit report in pastas>=1.5:

print(ml.fit_report(stderr=True))

To show the parameter uncertainty estimates in the results plot in pastas>=1.5:

ml.plots.results(stderr=True)

To show the parameter uncertainty estimates after ml.solve():

ml.solve(report="full")
@raoulcollenteur raoulcollenteur added the development Indicates development of new features label Apr 11, 2024
@raoulcollenteur raoulcollenteur added this to the 1.5 milestone Apr 11, 2024
@raoulcollenteur raoulcollenteur self-assigned this Apr 11, 2024
@raoulcollenteur raoulcollenteur pinned this issue Apr 11, 2024
@raoulcollenteur raoulcollenteur changed the title Noisemodel in ps.Model and noise argument in ml.solve are Deprecated as of Pastas 1.5 The noisemodel in ps.Model and noise argument in ml.solve are Deprecated as of Pastas 1.5 Apr 11, 2024
@mbakker7 mbakker7 changed the title The noisemodel in ps.Model and noise argument in ml.solve are Deprecated as of Pastas 1.5 The noisemodel keyword argument in ps.Model and noise keyword argument in ml.solve are Deprecated as of Pastas 1.5 Apr 11, 2024
@mbakker7 mbakker7 changed the title The noisemodel keyword argument in ps.Model and noise keyword argument in ml.solve are Deprecated as of Pastas 1.5 The noisemodel keyword argument in ps.Model and noise keyword argument in ml.solve are Deprecated as of Pastas 1.5 Apr 11, 2024
@rubencalje rubencalje unpinned this issue Apr 12, 2024
@rubencalje rubencalje pinned this issue Apr 12, 2024
@martinvonk martinvonk added the priority 0 high, deal with as soon as possible label Apr 12, 2024
@raoulcollenteur raoulcollenteur modified the milestones: 1.5, 1.6 Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development Indicates development of new features priority 0 high, deal with as soon as possible
Projects
None yet
Development

No branches or pull requests

3 participants