@virio-andreyana could you add a comment to the release notes?

@fneum do you agree with the changes? Note that this is a breaking change (which is however necessary to my opinion)

Hmmm, while I agree that this change very nicely separates resources from results, I see one significant drawback.

It considerably limits the use of shared_resources: true, as it is no longer possible to deviate from the shared resources before prepare_sector_network, where most scenario-dependent settings are made.

For instance, if you have two config files

snakemake ... --configfile config.scenario1.yaml
snakemake ... --configfile config.scenario2.yaml

that only differ in prepare sector networks, you can't do that with shared_resources: true anymore.

@fneum, thinking about it, I see the problem as very minor. Assuming that the configuration overrides only contain changes in the sector section, both the proposed version and the master branch would trigger the prepare_sector_network run on every snakemake call. The only difference is that the proposed version would not keep the result of prepare_sector_network in the resources folder, but would overwrite it once scenario2 runs. However, both calls would end up in separate result folders.

@fneum, thinking about it, I see the problem as very minor. Assuming that the configuration overrides only contain changes in the sector section, both the proposed version and the master branch would trigger the prepare_sector_network run on every snakemake call. The only difference is that the proposed version would not keep the result of prepare_sector_network in the resources folder, but would overwrite it once scenario2 runs. However, both calls would end up in separate result folders.

@fneum what do you think?

Yes, that's right, but only as long as you're not using --rerun-triggers=mtime, in which case snakemake will ignore changed params and not rerun prepare_sector_network.

In the old case, if the configfiles had different run names, prepare_sector_network would be triggered in any case.

If you have everything run until right before prepare_sector_network, and you do the following two calls without waiting for the other to finish first, you could also get interference. (I am doing this a lot).

snakemake ... --configfile config.scenario1.yaml --nolock
snakemake ... --configfile config.scenario2.yaml

(this is a smaller argument as you should soon be able to do this in one call with new scenario management)

Okay I see. Then, let's wait until we merge #724 and merge this one afterwards.