whitespace issues in sipnet.out #2266
Comments
|
Surely, someone else has run into this? I hit this, so far, ONLY at US-Me2......anyone else? |
|
my present "fix" is to switch from using PFT "boreal.coniferous" to " |
|
I remember encountering this error. It's been a while so I don't fully remember the details but I think my solution was to change the sipnet code itself allowing more digits in |
|
@istfer Thanks! @para2x trying to run the SDA code and this only occurs while I setup my sipnet runs, specifically when using the boreal.coniferous PFT. Were you able to run sites with that PFT? It seems whenever I try to use that PFT it fails with this issue. Perhaps you are using the model binary that @istfer created after updating the code as she mentions. |
|
Looks like the issue is like you said with coarseRootC specifically with the value |
|
@robkooper I suspect the issue is the parameterization of that PFT by pecan as I have been finding very large resulting params in SIPNET for roots with some pfts which probably then creates large root pools in sipnet. I do like the idea of just increasing the max....though that may just allow for more non-sensical numbers to spit out. |
Yes I do. I made my xmls based in Istems which we explicitly have a tag in xml which points to a version of SIPNET. |
|
Yeah this is a problem in general since now I am hitting this issue with my SDA test runs.... which in this case is not related to boreal.coniferous pft except, perhaps, because the parameter file for IC is based on Bartlet which IS a BC pft, @para2x? thoughts? |
|
I really don't know enough about the SIPNET code itself to make a suggestion here. |
|
No worries, @para2x my Q was more about how you are using the file https://github.com/PecanProject/pecan/blob/develop/modules/assim.sequential/inst/MultiSite-Exs/SDA/Bartlett.param which may have a param value the influences rootC that is causing really large values |
|
The only reason I use |
|
Someone at BU, could you tell me what your sipnet.c file looks like on lines 782-785, and 813? That is for BETY model ID number 1000000014. I want to see if this is a version where the output has been edited |
|
I think this is it the last variable name in the header after LAI is somehow cut, but it should be wood creation (and note that this might not be in other sipnet.c files, this was another tracker I created (I should've moved this under another sipnet version) |
|
Yeah, not the same in my sipnet.c, which is base r136 lines 787-840 |
|
I am going to first try increasing the output size for coarseRootC in sipnet.C to something like 9.2f; not sure we should increase it larger than that |
|
My gut feeling is that the write.table has a specific format for a floating point number (4.2%f is what it seams) and it results in an invalid file (still experimenting with this). |
|
@robkooper not sure I understand.....did you mean read.table? I changed to 9.4f and so far havent hit the error yet for the same site/pft combo |
|
that is, in the sipnet.C file for coarseRootC output variable |
|
Yeah . didnt help. I guess we just need to increase whitespace |
|
Trying to figure out how that file you pasted was created. Was this created by PEcAn? I tried to run the same run and generated the input file (https://pecan-docker.ncsa.illinois.edu/minio/dbfiles/CRUNCEP_SIPNET_site_0-763/) I don't see this problem in the file there. The numbers should be tab separated, not space. |
|
@bailsofhay FYI - this could be a cause for your mysterious SIPNET crashes. We should investigate this further |
|
Attempting to fix this by updating sipnet.C changed 4 to 8.8 |
|
also added trackers.LAI based on @istfer example |
|
I updated the header: |
|
OK so this has worked, except I am finding a disconnect between what PEcAn is providing for LAI in the nc files versus whats in the LAI column of the SIPNET output. I know this is an issue that @istfer and @mdietze has pointed out before but I forget how we should be addressing this. So 1) where is PEcAn getting LAI, is it instead calculating after the fact using other output data instead of assuming LAI is output from SIPNET? 2) should we be using the calculated LAI or the LAI spit out from SIPNET after adding in the trackers.LAI? 3) For SDA, which should we be using as this may be a reason for some of the strangeness in SIPNET runs So for example, here is 2010 LAI output in SIPET (from sipnet.out), between 0.2 and ~23 m2/m2 BUT here is PEcAN output LAI
@istfer have you used SIPNET LAI or PEcAn calculated LAI? |
|
OK, hmmm...also SIPNET LAI doesnt go to 0...often hovers around 0.1 or 0.2, but sometimes as high as 0.5 in the winter?! For a broadleaf site
|
|
I'll talk to @istem today to see if she remembers the fix for the signet vs pecan LAI values |
|
Here's a new issue I think is related to this since we changed the signet.c file Why are we getting -0.000 values for AGB???? |
|
That's round off. The real value is negative but > -0.0000000005. And the column headers are delimited, not aligned, so I don't think that's actually AGB |
|
Changing this to be sci notation now. Will update when fixed changing col 19 (gpp) to: %8.3e |
|
OK, so actually I made a mistake before. What I actually need to do is fix two different columns. The first to address the initial issue, that cols 8 and 9 sometimes merge because there isnt enough space and the second is the notation for GPP (col 19). |
|
@serbinsh It looks like we need to think about GPP a little bit more on top of the scientific notation issue also. I was looking through my signet.out file for the ensemble that failed, and there are no negatives in this run, but the column spacing gets messed up later in the year when the GPP values got from the 1's to the 10's digit place. Here's an example (the columns are the same as the example I posted above: |
|
@bailsofhay that isnt actually an issue and happens quite often. The issues I posted are more serious, 1) not enough spacing between columns such that two columns merge together and then PEcAN cant find the right number of columns. THe other issue is the possible rounding error for GPP. What you show above is extremely common for sipnet and to this point I have ignored it, but maybe this is an issue? I assumed not. Others? |
|
@serbinsh I think it is sort of wrong. If you look at some of the other columns where the number of digits before the decimal increase, the decimal numbers stay lined up, and the numbers to the left of the decimal place move. GPP is the opposite, where the digits to the left of the decimal are lined up, but the numbers to the right of the decimal are no longer lined up. You may be right that this isn't actually a problem, but it's possible it could be the problem that is crashing some runs. I'll look at some of the successful runs and see if they have the change with digits lining up, and if they exist in other files, then this probably isn't a problem |
|
@bailsofhay the reason I dont think its a problem is that PEcAn would fail if the number of columns didnt come out to 28 or line up. So look for that error in the logs, if you dont find it that isnt a problem |
|
Also I just update sipnet to use sci notation for GPP, that may also address this, Please try re-running your SDA |
|
kk Will do. I also checked some of the other runs and the have the digit changes like failed one, but all the spaces are there, so you're correct. that isn't an issue. |
|
This issue is stale because it has been open 365 days with no activity. |
Bug Description
Here is an interesting one:
in the above, you can see that in the raw sipnet.out for US-Me2 for dates 1980-2010 using CRUNCEP we get a collapse of whitespace which then crashes pecan's model2netCDF with this error:
Looks like it is "coarseRootC" based on the "names" of the output
To Reproduce
Run SIPNET (v136) at US-Me2 for dates 1980/01/01 to 2010/12/31 then try to parse the outputs to netCDF. For some reason, it does this in most but not all sipnet.out files for my run?
Not really sure how to proceed with this one.
The text was updated successfully, but these errors were encountered: