Covariance matrix not supported in IES? #260
-
|
For calibration using pilot points in layer 1, I am trying to reference covariance matrices in my pest control file after using PPCOV and ADDREG1 utilities. I have been advised to use the covariance matrices to guide parameter reasonableness based on spatial relationships between the points. The .mat files are cited alongside the names of regularisation observation groups. Error condition prevents further execution: This pest control file works when running PEST_HP_MKL with no issue. I've had a look through the gmdsi covmats tutorial document but haven't found anything that specifies any difference for referencing the cov matrices in the control file when using pestpp-ies as opposed to pest_hp_mkl. Is the use of the covariance matrices still advisable when using ies? Any help/advice would be appreciated. Thanks! |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 6 replies
-
|
If you want to supply a prior parameter covariance matrix for ies, it's done with the optional "parcov" argument. Can you try that? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks j, that works. I did not realise this could be used to specify parameter covariance for history-matching as well as uncertainty! |
Beta Was this translation helpful? Give feedback.
-
|
When supplying the cov matrix using parcov, I am now getting the following warning and pest does not move on to the second iteration:
I have not encountered this before and the number of observations with conflict is a large percentage of my calibration data set. Do you have any idea what might be causing prior data conflict? A single run using one of the sampled parameter sets returns reasonable simulated matches to the observation data (however there are 3 different observation group types included, with residuals of different scales - I have weighted these accordingly but am not sure I have done so correctly) |
Beta Was this translation helpful? Give feedback.
-
|
That's just a warning - ies will normally continue...if you are using windows, it looks like you might have paused execution by clicking on the cmd window? PDC is an interesting issue and it can only be detected with a prior Monte Carlo - essentially the Monte carlo simulation results don't cover the observed values. Since the prior ensemble represents maximum expected parameter uncertainty, if the prior Sim results don't cover the obs values, it might take some extreme parameters (or combinations) to force the results to reach those obs... |
Beta Was this translation helpful? Give feedback.
-
|
At this point I've trialed multiple versions, adjusting some of the obs weights and other ies flag options - they return different numbers of conflicts but the issue persists and pestpp does not make it beyond the 36 run round. The issue of Monte carlo simulation results not covering the observed values could be true for one of the observation groups (known condition of zero flow rate, rather than measured observations - this has been included as an observed flow of 0.001 m3/d observation target however most of the simulated rates seem to exceed ~500) so I plan to trial a pestpp with this removed. What confused me is that the PDC occurs for the other observation groups as well... Could it also be worth revising any of the ies settings such as ies_reg_factor(currently at 1.25) or specifying ies_accept_phi_fac(currently default)? |
Beta Was this translation helpful? Give feedback.
-
|
Can you elaborate on the 36 run round? I think that's the lambda testing runs with default settings? What happens after that? What does the rec file say? Ies shouldn't (hopefully) fail silently - there ought to be an error message. It's possible that all the lambda test runs failed, esp if there are lots of big residuals from those conflicts - these can cause really large and extreme parameter changes. If you want to see the conflicts yourself, just load up the 0.obs.csv file and plot histograms of the conflicted columns and compare that to the obsval in the control file. I think there is also a PDC summary csv file... |
Beta Was this translation helpful? Give feedback.
-
|
NPHINORED was set to 4 and it was unable to find lamda/line search factor to meet ies_accept_phi - did not save the text from cmd but it did give the error message on termination. This is a sample from the pdc summary csv. Can I specify something within the control file which will increase the standard deviations used for each observed and simulated obs? I think that would solve the issue and could be what is wrong with my setup! <style> </style>
|
Beta Was this translation helpful? Give feedback.
-
|
ok so ies didnt find a lambda+scale factor value that reduced phi acceptably for 4 testing cycles...usually with the GLM algorithm, if the expected search direction doesnt go down hill at all, this usually means something is goofed in the pest interface (at least for me!). The sim standard deviation is coming from the simulation results of the prior parameter ensemble - these are pretty narrow ranges! So if you want that to be wider, you need to add more parameters, increase parameter bounds, or both...the obs standard deviation is derived from the weights unless you tell ies otherwise (with an optional "standard_deviation" column in the obs data section of the control file if you are using a version 2 control file...). But I think the issue is with the lack of spread in the simulation results - just looking at |
Beta Was this translation helpful? Give feedback.
If you want to supply a prior parameter covariance matrix for ies, it's done with the optional "parcov" argument. Can you try that?