[QUESTION] Getting custom math masks/scale factors to work in GCHP

[lfreese]

I'm trying to set up a run in GCHP with country level scale factors that are math (cos) functions.

I followed the instructions shown here: https://hemco.readthedocs.io/en/latest/hco-ref-guide/more-examples.html?highlight=mask#scale-or-zero-emissions-with-a-rectangular-mask. But I'm having some issues with getting it to work. I took out the math functions so that it is just integer values as scale factors, and it still doesn't work. My impression is that its an error with how I have the scale factors/masks written in ExtData, and have changed formats around to match what the existing examples are and still can't get it to work.

A second component of this is that I ideally want the scale factor to be a math function such as is shown under 'Scale Factors' here: https://geos-chem.readthedocs.io/en/latest/gcc-guide/03-config/hemco-config.html. I'm not sure if that is possible/feasible when using a scalefactor.txt file (it seems like it should be but I haven't seen an example of it before, and am still stuck just getting a normal scale factor to work first).

I'm using GCHP v13.4, attached my ExtData, HEMCO_Config, allPEs, scalefactors.txt, and slurm output below. I turned on all of the debug flags in HEMCO_Config and for allPEs.
scalefactor.txt
ExtData.txt
slurm-509412.txt
allPEs.txt
HEMCO_Config.txt

[lizziel]

Hi @lfreese, do you also have the general run log (gchp.log) and the HEMCO.log files?

[lfreese]

Yes! Here is the gchp.log. There is no HEMCO.log (it didn't get that far).

gchp.txt

[lizziel]

Hi @lfreese, I think what you are trying to do should not be a problem for GCHP since it is all related to HEMCO masking and scaling, something done separate from MAPL I/O.

The main thing that pops out at me is you are inputting a text file. You need to make it a netcdf file.

#==============================================================================
# --- Country mask factor ---
#==============================================================================
9985 SCALE_COUNTRY /net/fs11/d0/emfreese/GCrundirs/IRFruns/stretch_cos/scalefactor.txt  - - - xy count 1

Also, a few tips for your debug runs. First, changing to a low resolution will make the run faster. Second, change this line in HEMCO_Config.rc:
Logfile: HEMCO.log
to this:
Logfile: *
This will send all HEMCO prints to the main log rather than HEMCO.log. It is default in 14.0 but not yet there in 13.4. It can make finding the problems in HEMCO easier.

[lizziel]

I am transferring this issue to HEMCO.

[lfreese]

Thanks @lizziel-- I'm testing out the edits you suggested today and will see how they go.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. If there are no updates within 7 days it will be closed. You can add the "never stale" tag to prevent the Stale bot from closing this issue.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. If there are no updates within 7 days it will be closed. You can add the "never stale" tag to prevent the Stale bot from closing this issue.

[lizziel]

Hi @lfreese, I will close out this issue but feel free to reopen if you are still working on this.

[lfreese]

Coming back to this (apologies this was a long delay)-- I found some errors that I had made in setting up the .txt file for the scalefactors and for the way it was read into ExtData.rc and HEMCO_Config.rc (mostly based on GCHP issue 299). After fixing those, I am able to run a simulation in GCHP where the emissions of BC over Indonesia are doubled successfully, so I know the scalefactors.txt file should be reading in properly. However, I am not able to get the math function to work still. I've modified the scalefactors.txt file to be read in with a timestamp of * given that this is a math function that varies by hour. I've used the HEMCO mathematical expressions guide here to set this up.

The lines being used for this are below, and the files themselves are attached. I get the error statement of forrtl: severe (59): list-directed I/O syntax error, unit -5, file Internal List-Directed Read

HEMCO_Config:

* SCALE_BC_COUNTRY ./countrymask_0.1x0.1.nc CountryID 2000/1/1/* C xy count * - 1 1

9985 SCALE_COUNTRY ./scalefactor.txt  - * - xy count 1

(I've also tried:

* SCALE_BC_COUNTRY ./countrymask_0.1x0.1.nc CountryID 2000/1/1/1 C xy count * - 1 1

9985 SCALE_COUNTRY ./scalefactor.txt  - - - xy count 1

in combination with the above formats)

ExtData:
SCALE_BC_COUNTRY count N Y - none none CountryID ./countrymask_0.1x0.1.nc

scalefactors.txt:

# Country mask field name
SCALE_BC_COUNTRY

# Country data
# Name   | ID  | Scale factor
DEFAULT    0     1.0
INDONESIA  360   MATH:(2+cos(HH/6*PI))/2

I've attached the gchp.log, slurm errors, HEMCO.log, and have turned on all of the highest debug printouts. It is failing right at the spot where it should be reading in the MATH scale factor, so it definitely is the MATH component. I'm wondering if I'm having it read in the hourly scale factor in a way that isn't compatible?

ExtData.txt
gchp.txt
HEMCO_Config.txt
HEMCO.txt
scalefactor.txt
slurm-102887.txt
warnings_and_errors.txt

[Twize]

Hi @lfreese, the error forrtl: severe (59): list-directed I/O syntax error, unit -5, file Internal List-Directed Read sounds to me like the formats of the resulting values from the math expression may be incompatible with the scale factor field (or possibly the MATH expression is returning mixed formats like ints and floats). I'm not sure if there is a way to print out the result of the math expression, but if there is, I would check that.

Another thought is that you might not be able to put math expressions in the scale_factors.txt file, since it may not be directly read by HEMCO in the same way that the HEMCO_Config.rc file is, but rather HEMCO just grabs the values out of it. This is just a thought though, I have no idea if this is true or not. One possible way around this would be to use the mask file for indonesia to isolate the region, and then apply the math expression separately in the HEMCO_Config.rc. I have never tried this myself though, so not entirely sure how feasible it is.

As a quick update to my issue geoschem/GCHP#299, I was able to get the scaling working in my GCHP simulation in the end and it looked pretty good, so its definitely possible to apply scale factors this way in GCHP (however I never used math expressions).

[lizziel]

Hi @lfreese, would you be able to send me your netcdf file? I can try to reproduce the problem to figure out what is happening.

[lfreese]

Thanks for the help both! I'm using the classic country mask found here: (http://geoschemdata.wustl.edu/ExtData/HEMCO/MASKS/v2014-07/countrymask_0.1x0.1.nc)

@Twize I was able to get this to work for just an integer scaling, so I agree this technically should work and it is something with the reading in of MATH.

For what it's worth, I also tried to just hard code in the values for the sin wave as a daily changing scaler (eg: 2.0/2.1/2.5/3.1/2.5/2.1/2.0) so that it rotated over seven days, but that also fails with the same error as when a MATH object is put into the scalefactor.txt file. So it seems to definitely be something with how GCHP reads in more complex scale methods with nonnumerical values.

[lizziel]

Okay, I'll try to reproduce this now and let you know what I find. Thanks for your patience!

[lizziel]

Hi @lfreese, many apologies this issue keeps getting pushed back. Do you still have this issue?

[lfreese]

I haven't yet solved it-- fine to be on the backburner, but does seem worth figuring out in the long run! The only way I can get this to work is by setting individual scale values and changing them after single day runs (did this to test if even single value scales worked, and they do). The method of doing weekly scale values separated (eg: 3 / 5 / ...) doesn't work, so it seems to be an issue with non-numerical symbols?

[lizziel]

Okay, I will keep this open. Could you provide here your ExtData.rc and HEMCO_Config.rc for two different scenarios: (1) the case that does not work, and (2) the case that does. This will help me hone in on it faster.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. If there are no updates within 7 days it will be closed. You can add the "never stale" tag to prevent the Stale bot from closing this issue.