build metadata must be re-executed when model is rebuild

[timtroendle]

Build metadata has no inputs and is thus build initially and only once. This is no problem when starting from a clean working directory.

When the working directory is not clean and the model is rebuild, that may be a problem. As an example, one could build euro-calliope v1.0 and receive the correct build metadata. One could then update the configuration parameters and re-execute the model, without build metadata being updated leading to an inconsistent build-metadata.yaml. We need to automatically trigger a rebuild in these cases.

[timtroendle]

This is related to snakemake/snakemake#976.

[brynpickering]

Can the metadata not take one of the model.done files as input, thus triggering its build as the final thing in any build process?

[timtroendle]

Good idea. Currently, that's not possible because the metadata is part of the model. But we could change that and yes, that sounds like a good idea.

The disadvantage is that we likely have to move metadata then into the resolution-specific folder and have several copies of it. Otherwise the trigger is weird and can become inconsistent.

[brynpickering]

How about we change it around so that rule all is focussed on the metadata?

rule all:
    message: "Generate euro-calliope pre-built models and run tests."
    input: "build/model/build-metadata.yaml"

rule build_metadata:
    message: "Generate build metadata."
    input:
        script = script_dir + "metadata.py",
        "build/logs/continental/model.done",
        "build/logs/national/model.done",
        "build/logs/regional/model.done",
        "build/logs/continental/test-report.html",
        "build/logs/national/test-report.html",
    params:
        config = config,
        version = __version__
    output: "build/model/build-metadata.yaml"
    conda: "envs/default.yaml"
    script: "scripts/metadata.py"

of course, build metadata would then not be part of the rule model.

[timtroendle]

Makes sense. I would not move tests because they are unrelated. But that's partly subjective.