Issues with LUC files - round 2

[dariak-bsc]

Hi all,

Errors & warnings found with the file checker - preliminary results:

1. (Warning) Some attributes are missing, e.g. dataset_category, grid_label, source_id, table_id, target_mip
   
2. (Error) There are missing values in the data
3. (Error) In the reference files, latitude and longitude are called 'lat' and 'lon' correspondingly, but in the new files they are called 'latitude' and 'longitude'

@etiennesky

[coroa]

Hi @dariak-bsc ,

thank you for submitting those issues with your checker!

@pascal-sauer @tscheypidi FYI and please comment!

[ShraddhaGupta28]

Hi all,
The reference land use files have annual time resolution for the LU states, transitions and management, however the RESCUE land use scenarios came with time gaps of 5 years/10 years. The model (JSBACH) of course requires the files to have an annual time resolution. Can we expect the next version of the land use files to come with annual time resolution, or do we have to solve the issue ourselves by interpolating ?
Thanks !

[pascal-sauer]

Hi and sorry for the late reply.

(Warning) Some attributes are missing, e.g. dataset_category, grid_label, source_id, table_id, target_mip

What are the expected values for these attributes?

(Error) There are missing values in the data

I'll double check.

(Error) In the reference files, latitude and longitude are called 'lat' and 'lon' correspondingly, but in the new files they are called 'latitude' and 'longitude'

Should be possible to just rename these, I'll check.

[pascal-sauer]

Hi @ShraddhaGupta28, the magpie model produces output with these time steps (5/10 years). Additional time steps would always simply be interpolated, which massively increases files sizes and leads to (admittedly fixable) memory problems on our side. So for the iterating phase where we still expect bug fixes and changes in response to feedback we would stick with the 5/10 year time steps, and kindly ask you to do the interpolation on your end.

[ShraddhaGupta28]

Hi @pascal-sauer,

There is also a problem regarding the generic grid coordinates when comparing the reference files and the RESCUE land use files. This causes an issue when remapping the new files using CDO for preparing the ESM input cdo gencon (Abort): Unsupported generic coordinates !

On looking closer using cdo griddes <infile.nc>, the following difference is found between the reference files and the new files:

• For the reference files.

cdi  warning (cdfScanVarAttr): NetCDF: Variable not found - bounds_time
cdi  warning (cdfScanVarAttr): NetCDF: Variable not found - bounds_lat
cdi  warning (cdfScanVarAttr): NetCDF: Variable not found - bounds_lon
#
# gridID 1
#
gridtype  = lonlat
gridsize  = 1036800
xsize     = 1440
ysize     = 720
xname     = lon
xlongname = "longitude"
xunits    = "degrees_east"
yname     = lat
ylongname = "latitude"
yunits    = "degrees_north"
xfirst    = -179.875
xinc      = 0.25
yfirst    = 89.875
yinc      = -0.25
#
# gridID 2
#
gridtype  = generic
gridsize  = 1440
xsize     = 2
ysize     = 720
yname     = lat
ylongname = "latitude"
yunits    = "degrees_north"
yfirst    = 89.875
yinc      = -0.25
#
# gridID 3
#
gridtype  = generic
gridsize  = 2880
xsize     = 1440
ysize     = 2
xname     = lon
xlongname = "longitude"
xunits    = "degrees_east"
xfirst    = -179.875
xinc      = 0.25
#
# gridID 4
#
gridtype  = generic
gridsize  = 2
xsize     = 2
cdo    griddes: Processed 121 variables [0.13s 33MB]

• For the new files:

cdi  warning (cdf_set_dim): Inconsistent dimension definition for latitude! dimid = 0;  type = 1;  newtype = 2
cdi  warning (cdf_set_dim): Inconsistent dimension definition for longitude! dimid = 0;  type = 2;  newtype = 1
#
# gridID 1
#
gridtype  = lonlat
gridsize  = 1036800
xsize     = 1440
ysize     = 720
xname     = longitude
xlongname = "longitude"
xunits    = "degrees_east"
yname     = latitude
ylongname = "latitude"
yunits    = "degrees_north"
xfirst    = -179.875
xinc      = 0.25
yfirst    = 89.875
yinc      = -0.25
cdo    griddes: Processed 12 variables [0.22s 33MB]

[pascal-sauer]

Sure, I can add the bounds variables

[ShraddhaGupta28]

Hi @pascal-sauer,
I think the problem is not about the missing bounds because the bounds are missing in the reference files too. But rather the problem is something about the missing generic grids latitude and longitude. In my previous comment, you see that cdo griddes on the reference file yields 4 gridIDs (see 1st bullet) out of which one is of gridtype latlon while the other three are gridtype generic, while for the new RESCUE Land use files (2nd bullet in the previous comment) one gets only 1 gridID of gridtype latlon. I think these generic gridtypes for lat and lon are important for CDO remapping commands.

[pascal-sauer]

The bounds in the reference files are actually present, but not called like the attributes suggest. They are called lat_bounds instead of bounds_lat etc. I'll now include them as bounds_lat as the attribute indicates.

[pascal-sauer]

Hm I'll check what cdogriddes returns. Maybe renaming to lon/lat as opposed to longitude/latitude already helps. I did set all attributes which are set in the reference files now, so I hope cdo will treat the files exactly like the reference files now. (I did not recreate/upload the dataset yet, but I already put the changes in our pipeline in place)

[dariak-bsc]

Hi @pascal-sauer ,
I checked the latest data /forcings/land_use/2024-04-25:

• there are some missing values but in fact it seems that the mask for the data is a little different from the mask of the reference files (I am going to make a plot with these points as discussed with @etiennesky );
• multiple-management and multiple-transition files don't have units.

[pascal-sauer]

Thanks for checking Daria, I'll add units and reupload. On the mask with missing values: My understanding is that you check with Etienne if this is fine, and come back to me in case we need to change something, yes?

[dariak-bsc]

Hi @pascal-sauer and @etiennesky , I think maybe we could discuss the mask and the missing values in tomorrow's meeting.
(I ran the checker with the mask taken not from the reference file but from the corresponding variable in the file which is being checked, e.g. if I am checking the variable c3ann in the file multiple-states_input4MIPs_landState_RESCUE_PIK-MAgPIE-4-7-C-SSP2EU-NPi-2024.4.25_gn_2015-2100.nc then I am taking the missing values in c3ann at the first timestep as the mask to check the following timesteps. And in this case, I am not getting any missing values at the following timesteps. When I take the mask from the reference file, then I get missing values.)

@pascal-sauer could you please confirm that the variables fharv_c3per, fharv_c4per, flood, combf in multiple-management files are not required?

[pascal-sauer]

Yes, I can confirm we agreed that fharv_c3per, fharv_c4per, flood, combf are not required.

[pascal-sauer]

I uploaded all files again with units. While I was working on the units I replaced unit "1" with "share of X" to make it clearer to which other variable a relative variable relates.