Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: checkpoints inside modules are overwritten #2843

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

fgvieira
Copy link
Contributor

@fgvieira fgvieira commented Apr 24, 2024

Description

When including two modules with checkpoints (or the same module twice with different prefixes), only the checkpoint of the last loaded module is used.

In the example here, the checkpoint somestep is run twice the module test_b (under prefix b) while it should be run one for each prefix (as exemplified under expected-results).

Assuming unrestricted shared filesystem usage.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job                 count
----------------  -------
all                     1
test_a_aggregate        1
test_b_aggregate        1
test_b_somestep         2
total                   5

Select jobs to execute...
Execute 1 jobs...

[Wed Apr 24 13:36:30 2024]
localcheckpoint test_b_somestep:
    output: b/somestep/a.txt
    jobid: 2
    reason: Missing output files: <TBD>
    wildcards: sample=a
    resources: tmpdir=/tmp
DAG of jobs will be updated after completion.

[Wed Apr 24 13:36:31 2024]
Finished job 2.
1 of 5 steps (20%) done
Select jobs to execute...
Execute 1 jobs...

[Wed Apr 24 13:36:31 2024]
localrule test_a_alt_intermediate:
    output: a/alt/a.txt
    jobid: 8
    reason: Missing output files: a/alt/a.txt
    wildcards: sample=a
    resources: tmpdir=/tmp

[Wed Apr 24 13:36:31 2024]
Finished job 8.
2 of 6 steps (33%) done
Select jobs to execute...
Execute 1 jobs...

[Wed Apr 24 13:36:31 2024]
localcheckpoint test_b_somestep:
    output: b/somestep/b.txt
    jobid: 4
    reason: Missing output files: <TBD>
    wildcards: sample=b
    resources: tmpdir=/tmp
DAG of jobs will be updated after completion.
[...]

QC

  • The PR contains a test case for the changes or the changes are already covered by an existing test case.
  • The documentation (docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).

Copy link

sonarcloud bot commented Apr 24, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

@fgvieira fgvieira added bug Something isn't working help wanted Extra attention is needed labels Apr 24, 2024
@fgvieira fgvieira changed the title fix: checkpoints inside modules are oerwritten (only the last is used) fix: checkpoints inside modules are overwritten Apr 24, 2024
@johanneskoester johanneskoester self-assigned this Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants