data catalog related error

[Safa136]

I got this error:

Note: It was working but today it gives me this error.

[Safa136]

The main error is: mapping values are not allowed here.

Although it was working yesterday:

[Safa136]

Please help with this issue!

[hboisgon]

Hi @Safa136
Unfortunately without the file I cannot really know what is wrong. In your error message it says:

Can you check or else share the file with us.
Good practice is to provide a minimum reproducible example for us to help you out (ie hydromt and hydromt-wflow version, command line used, in your case data catalog where the error seems to be)

[Safa136]

Hi hboisgon,

Thank you very much for responding to my messages.

I found out that I have an Indentation error in the raw specified in the YAML file.

I have a new question about the soil grids.

I downloaded the data from "https://files.isric.org/soilgrids/former/2017-03-10/data/" but could not find the soil thickness layer. I will be grateful if you help me with this issue.

the data that I downloaded:

1. BLDFIE_M_sl(1 to 7)
2. CLYPPT_M_sl(1 to 7)
3. OCDENS_M_sl(1 to 7)
4. PHIHOX_M_sl(1 to 7)
5. SLTPPT_M_sl(1 to 7)
6. SNDPPT_M_sl(1 to 7)
7. TAXOUSDA_250m_ll

[verseve]

For the soil thickness you can use BDRICM_M_250m_ll.tif from soilgrids. Note that for soil thickness the European Soil Database (ESDB) is also used (field IL (impermeable layer)) in hydromt_wflow as a source dataset. ESDB is the preferred dataset, and otherwise soilgrids is used.

[Safa136]

Thank you very much Willem van Verseveld for your prompt reply!

[Safa136]

Hi,

I have some questions regarding the setup of the wflow:

1. I want to set up my model for the time period of 1/12/2015 to 31/12/2015. Where should I change the date? Is it in the wflow_sbm.toml:
   casename = "wflow_sbm"
   calendar = "proleptic_gregorian"
   starttime = 2010-02-02T00:00:00
   endtime = 2010-02-15T00:00:00
   time_units = "days since 1900-01-01 00:00:00"
   timestepsecs = 86400

or in my configuration file (wflow_build.ini):
[setup_config] # options parsed to wflow ini file

.
starttime = 2010-01-01T00:00:00
endtime = 2010-03-31T00:00:00
timestepsecs = 86400
input.path_forcing = inmaps-era5-2010.nc

2. What is the (input.path_forcing) mean exactly? does it mean I should provide the forcing data (precipitation and temperature) to be in one NetCDF file and put it here? or is it just the name of the output file where the data will be stored? because I have already provided the forcing data in the configuration file through the data catalog.

3. I am always getting this error or warning (build - model_api - ERROR - missing required argument cfdict for setup_config). I believe it is related to the toml file., but I am not sure where to specify that.

Many thanks in advance!

[verseve]

1. You should change the dates in the hydromt_wflow configuration file (wflow_build.ini) file. The starttime and endtime are used to extract the correct time period from the input forcing datasets used by hydromt, and are also written to the Wflow settings TOML file. You want to setup the Wflow model for a rather short time period, we recommend to set it up for a longer time period (years), because you also need a period to spin-up the model to get the correct initial conditions. Also when running a Wflow model, you can always change the starttime and endtime in the Wflow settings TOML file, to run for a smaller period (subset of the input forcing data).

2. The input.path_forcing is the file name used by hydromt_wflow to write gridded forcing data at the Wflow model resolution.

The source of the precipitation data should be specified below the section [setup_precip_forcing], and source of temperature and potential evapotranspiration data should be specified below the section [setup_temp_pet_forcing] in the wflow_build.ini file. These sources should be linked to your own data catalog (yaml file).

See also:
https://deltares.github.io/hydromt_wflow/latest/user_guide/wflow_build.html#wflow-build
https://deltares.github.io/hydromt/latest/user_guide/data_prepare_cat.html
https://deltares.github.io/hydromt/latest/user_guide/data_existing_cat.html

[Safa136]

Thank you so much Willem van Verseveld for your prompt and helpful responses! I really appreciate your support.

[verseve]

3. I am always getting this error or warning (build - model_api - ERROR - missing required argument cfdict for setup_config). I believe it is related to the toml file., but I am not sure where to specify that.

Without the TOML file and a bit more info (ie hydromt and hydromt-wflow version, command line used) so we can reproduce it, it is difficult to help you out with this question/issue. For a first complete Wflow model setup, a TOML file is not required.

[Safa136]

hydromt.log
I attached my hyromt.log file and also my configuration file.

[Safa136]

The configuration file is not supported here to be attached. anyway the error is line number 6, but it doesnot affect the setup process.

[Safa136]

I have more questions about the data. I will be grateful if you can help on them.

1. ERA5 data:
   I downloaded the hourly data on single levels from here (https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=form), and then I developed a python code to resample the data to daily frequency.
   The data are:
   a) Maximum 2m temperature since previous post-processing
   b) Minimum 2m temperature since previous post-processing
   c) 2m temperature
   d) Mean sea level pressure
   e) Surface solar radiation downwards
   f) TOA incident solar radiation
   g) Total precipitation

1.1. My question or concern is that I see in the example of deltares data catalog in era5 data that it is on pressure levels. but my data on single level. is that a problem?
1.2. Should I need (era5_orography) as the data of geopotential according to pressure levels.

2. MODIS LAI data:
   I downloaded the data from https://search.earthdata.nasa.gov/ for selected days in .hdf file, then I developed a code myself to resample the data to daily frequency using interpolation etc.. (as the model run in a daily time step) and after that I converted my file to one netcdf file (modis.nc) and describe it in the my catalog.

2.1. My question does this create a problem or it is the ok to do that? because I see in a comment here (We use monthly climatology data for LAI) in issue #156. Do we use monthly or daily? does the model itself will resample the data from montly to daily? I am a littile bit confused.

[verseve]

hydromt.log I attached my hyromt.log file and also my configuration file.

@hboisgon or @JoostBuitink are you perhaps familiar with the ERROR message?

[verseve]

I have more questions about the data. I will be grateful if you can help on them.

1. ERA5 data:
   I downloaded the hourly data on single levels from here (https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=form), and then I developed a python code to resample the data to daily frequency.
   1.1. My question or concern is that I see in the example of deltares data catalog in era5 data that it is on pressure levels. but my data on single level. is that a problem?

No, this is not a problem. Also the Deltares data catalog uses single level, so I guess this is a typo.

1.2. Should I need (era5_orography) as the data of geopotential according to pressure levels.

The orography data is used for the lapse rate correction (era5 pressure and temperature data) using the Wflow DEM (model resolution). I would recommend to include the lapse rate correction.

2. MODIS LAI data:
   I downloaded the data from https://search.earthdata.nasa.gov/ for selected days in .hdf file, then I developed a code myself to resample the data to daily frequency using interpolation etc.. (as the model run in a daily time step) and after that I converted my file to one netcdf file (modis.nc) and describe it in the my catalog.
   2.1. My question does this create a problem or it is the ok to do that? because I see in a comment here (We use monthly climatology data for LAI) in issue Bug: deriving basin in setup_basemaps does not work without basin_index_fn #156. Do we use monthly or daily? does the model itself will resample the data from montly to daily? I am a littile bit confused.

In the Deltares catalog we use monthly LAI data, this is climatology data. So it is average data for 12 months computed over a long period. For an example you could check the staticmaps.nc file in /hydromt_wflow/examples/wflow_piave_subbasin (variable LAI). Wflow.jl will resample to the model timestep.
If you prefer you can also use LAI climatology data at a daily timestep, computed over a long period. For what period did you download LAI data?

[Safa136]

If you prefer you can also use LAI climatology data at a daily timestep, computed over a long period. For what period did you download LAI data?
I downloaded the data just for 2 month.
but I want to setup my model from 1/12005 to 31/12/2022.

[verseve]

I downloaded the data just for 2 month. but I want to setup my model from 1/12005 to 31/12/2022.

2 months is too short for a climatological average, my recommendation is to use at least 10 years.

[Safa136]

I will take your advice and will download the data for 15 years.

Thank you very much for your reply and support.

[verseve]

Hi @Safa136, I just e-mailed the LAI data for Ireland that you should be able to use directly for the setup of your Wflow model, as an intermediate solution for related issue #157.

[Safa136]

Hi @verseve,

Thank you very much for sending the LAI data. I have a question regarding the data.

I believe the 12 rasters (.tiff) represent the average monthly data for the 12 months. Does the data average over the period '2003-01-01' to '2017-12-31' as written in the Deltares data catalog? or a longer period (updated) because I want to use the data to set up my model from 1/1/2005 to 31/12/2022.

[verseve]

Yes, the data is the average over the period 2003-01-01 to 2017-12-31 and it has not been updated (yet). I don't think it will affect the modeling results significantly. Anyway, no problem to also provide the LAI data for the period 2005-01-01 to 2022-12-31.

[Safa136]

I wanted to double-check, but I also believe that it won't have a significant impact on the results.

Thank you very much @verseve for your prompt response and support.

[Safa136]

Hi,

How to let hydromt-wflow read the Path to a mapping csv file from Landuse?

I write the LULC in my data catalog here

and I refer to it in the configuration file here

the hydromt-wflow process is completed as here in hydromt.log
hydromt.log

However, it is written that the variables of lulcmaps were derived as here

but their values are NAN because (I think hydromt-wflow did not access the mapping csv file)
corine_mapping.csv

I tried to run the model in Julia and I ensured that the variables related to the LULCmaps ['landuse', 'Kext', 'N', 'PathFrac', 'RootingDepth', 'Sl', 'Swood', 'WaterFrac']) are NAN.

I double checked by reading the generated staticmaps.nc in MATLAB and found these variables are NAN.

Thank you for helping me in advance!

[verseve]

but their values are NAN because (I think hydromt-wflow did not access the mapping csv file

And the landuse map in staticmaps.nc (wflow_landuse) looks correct?

If you do not provide a path to a mapping csv file in the configuration file (lulc_mapping_fn), hydromt_wflow will use the default path of your setup as part of the hydromt_wflow installation, see also: https://github.com/Deltares/hydromt_wflow/blob/main/hydromt_wflow/wflow.py#L794. So, in your case this should resolve in \hydromt_wflow\data\lulc\corine_mapping.csv.

[Safa136]

wflow_landuse values are also NAN.

[verseve]

wflow_landuse values are also NAN.

And what about the source file Corine_Europe.tif, is it containing correct values? This source file is just resampled (nearest neighbor) to the wflow_landuse map.

[Safa136]

Hi @verseve

I checked the source data Corine_Europe.tif. and to be sure of my data I updated the source data in the catalog with that I downloaded from the website without clipping or anything.

Still, I got NAN for landuse and all LULC parameters in staticmaps.nc

[Safa136]

I am thinking of two things:

1. maybe the projection of the data is not matching because the projected coordinate system of this tiff is (ETRS 1989 LAEA) not (WGS 1984).
2. Or the hydromt-wflow can not access the mapping csv file. but according to this:

it should be accessed . here is the path of the mapping csv file.

[verseve]

I don't think it is the projection, hydromt can work with different projections.

Could you share a minimal reproducible example (a snippet of the data, hydromt-wflow configuration (ini) file, and data catalog (yml) file) so we can easier check what is causing this issue?

[Safa136]

I emailed you hydromt-wflow configuration (ini) file, and data catalog (yml) file).

[verseve]

I just noticed that the corine land cover map you are using (version 2020_20u1) is different from the corine land cover map in the deltares catalog (version 20). Version 2020_20u1 contains values 1-45 and 48 (nodata), while version 20 contains the actual CLC code (111, 112, 121,....,522, 523, 999 (nodata)).

To make the mapping compatible with the 2020_20u version you have to change the first column in your corine_mapping.csv file with 1-45 and 48.

[Safa136]

Thank you very much @verseve. Indeed that was the problem. I updated the corine_mapping.csv file and now landuse and lulc parameters have values!