Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use snakemake.params to auto re-run due to config changes #296

Closed
wants to merge 7 commits into from
Closed

Conversation

fneum
Copy link
Member

@fneum fneum commented Dec 4, 2021

Closes # (if applicable).

Changes proposed in this Pull Request

Use the params: section in rule definition to keep track of changed settings in config.yaml.

https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#non-file-parameters-for-rules

The goal is that rules for which parameters have changed rerun automatically.

This can currently be done with:

https://snakemake.readthedocs.io/en/stable/project_info/faq.html#how-do-i-trigger-re-runs-for-rules-with-updated-code-or-parameters

snakemake -n -R `snakemake --list-params-changes`

or

snakemake -R $(snakemake --list-params-changes)

This will be even more convenient, once this snakemake PR is released:

snakemake/snakemake#1107

snakemake --rerun-params-changed

or

snakemake --rerun-changed

(which would also include code changes)

It seems to work very nicely to pass lists/dicts as params, which would help us keep config "blocks".

This PR is not complete yet, but a proof-of-concept for a selection of config settings. We can discuss from here and complete later.

Checklist

  • I tested my contribution locally and it seems to work fine.
  • Code and workflow changes are sufficiently documented.
  • Newly introduced dependencies are added to envs/environment.yaml and envs/environment.docs.yaml.
  • Changes in configuration options are added in all of config.default.yaml, config.tutorial.yaml, and test/config.test1.yaml.
  • Changes in configuration options are also documented in doc/configtables/*.csv and line references are adjusted in doc/configuration.rst and doc/tutorial.rst.
  • A note for the release notes doc/release_notes.rst is amended in the format of previous release notes.

@euronion euronion added this to the Release 0.5 milestone Jan 10, 2022
@fneum fneum removed this from the Release 0.5 milestone Jan 13, 2022
@fneum fneum marked this pull request as ready for review January 31, 2022 18:19
fneum added a commit that referenced this pull request Mar 6, 2023
@virio-andreyana virio-andreyana mentioned this pull request May 11, 2023
72 tasks
@fneum
Copy link
Member Author

fneum commented Jan 2, 2024

superseded by #663

@fneum fneum closed this Jan 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants