--until not working with checkpoints

[nkt97]

Hello,

I cannot use --until for rules between a checkpoint and the rule where the aggregation happens. If I use --until, it says Nothing to be done. If I do not use --until, everything works perfectly fine.

Here is a minimal example, where snakemake --cores all --until r2 does not work, even after running rule r1 beforehand.

rule all:
    input:
        'agg'
        
checkpoint r1:
    output:
        directory('B')
    shell:
        'mkdir -p B && touch B/1 && touch B/2 && touch B/3'
        
rule r2:
    input:
        'B/{i}'
    output:
        'C{i}'
    shell:
        'touch {output}'
        
        
def aggregate_input(wildcards):
    checkpoint_output = checkpoints.r1.get(**wildcards).output
    exp = expand('C{i}', i=glob_wildcards( 'B/{i}').i)
    return exp
    
rule r3:
    input:
        aggregate_input
    output:
        'agg'
    shell:
        'touch {output}'

I hope you can help me. Thank you very much in advance!
Khoa

[phspo]

can confirm and reproduce this

[cademirch]

The first issue makes sense and I believe is expected behavior. If r1 has not been run, r2 is not in the DAG yet, and thus running until it is not possible. It is odd that --until r2 does not work when r1 has already been run though.

Edit: Based on the help message of until:

"Runs the pipeline until it reaches the specified rules or "
"files. Only runs jobs that are dependencies of the specified "
"rule or files, does not run sibling DAGs. "

I guess the second situation kinda makes sense too, since r3 needs to be run to trigger the DAG re-evaluation. It is a bit odd since if you put a print statement in the aggregate_input function, the function is being run.

[nkt97]

Thank you for your answer! Still, would it be possible to add this functionality to --until somehow?

[cademirch]

I'm not sure. But if you figure out how to fix this, you should open a PR!