General options for a DataStructureDefinition

[danielhuppmann]

While thinking about how to implement a list of countries in a project repository (ideally drawn directly from the pycountry package), I had the following idea:

In the definitions folder, there can be a config.yaml file with "general" specifications, e.g.

- region:
    country: true
- time:
    datetime: true
...

Alternatively (or in addition), each definition/<dimension> folder could have such a config file.

The behavior would be as follows:

1. If region -> country == True, add all country names and ISO3-codes from the pycountry package to the RegionCodeList.
2. If time -> datetime == True, allow timeseries data with a datetime time domain. If false (default), fail the validate() method (see Validation of datetime format #54)

Any thoughts @phackstock?

[phackstock]

I like the idea, smells like a pydantic specification to me ;)
Jokes aside though, I think having one file in definitions/ called general.yaml is a good idea. As long as it does not get exceedingly big, I'd stay clear of having individual files for each dimension. It should be easier to have all the DataStructureDefinition configuration in a single spot.
One question regarding:

If region -> country == True, add all country names and ISO3-codes from the pycountry package to the RegionCodeList.

Would that be an at-runtime thing or written into the yaml files?

[danielhuppmann]

I would fill that at run-time - for the list of countries, this might still be possible (but duplicating) to create a yaml from the pycountry package. But for datetime-validation, writing a yaml would be pretty much pointless...

[phackstock]

Right, makes sense to me.