Ancient keyword causes incorrect scheduling order

[kernco]

Snakemake version
6.1.0

Describe the bug
In some situations, putting ancient() around input files causes rules to be run before the input files they need exist.

Minimal example
Here is a minimal example Snakefile:

rule all: 
        input: 'Step3.done'

rule Preprocess:
        input: 
        output: 'Preprocess.done'
        shell: ''

rule Step1:
    input: ancient('Preprocess.done')
        output: 'Step1_{i}.done'
        shell: ''

rule Step2:
        input: expand('Step1_{g}.done', g=list(range(3)))
        output: 'Step2.done'
        shell: ''

rule Step3:
        input: ancient('Preprocess.done'),'Step2.done'
        output: 'Step3.done'
        shell: ''

Doing a dry run (snakemake -n) results in this output:

[Wed Apr  7 22:45:37 2021]
rule Step1:
    input: Preprocess.done
    output: Step1_1.done
    jobid: 5
    wildcards: i=1

[Wed Apr  7 22:45:37 2021]
rule Preprocess:
    output: Preprocess.done
    jobid: 2

[Wed Apr  7 22:45:37 2021]
rule Step1:
    input: Preprocess.done
    output: Step1_0.done
    jobid: 4
    wildcards: i=0

[Wed Apr  7 22:45:37 2021]
rule Step1:
    input: Preprocess.done
    output: Step1_2.done
    jobid: 6
    wildcards: i=2

[Wed Apr  7 22:45:37 2021]
rule Step2:
    input: Step1_0.done, Step1_1.done, Step1_2.done
    output: Step2.done
    jobid: 3

[Wed Apr  7 22:45:37 2021]
rule Step3:
    input: Preprocess.done, Step2.done
    output: Step3.done
    jobid: 1

[Wed Apr  7 22:45:37 2021]
localrule all:
    input: Step3.done
    jobid: 0

When the DAG is built, it doesn't seem to recognize that Step1 cannot be run before Preprocess, and instead schedules them to run simultaneously. This can be seen by one of the Step1 jobs appearing before the Preprocess job.

Removing the use of ancient from Step1 results in the correct scheduling. This workflow works on older version of Snakemake, but at some point it stopped working. I'm not sure about the exact version where it stopped working, but I know that it works in version 5.12.0.

[yuefu-jiang]

I observed the same issue under 5.32.2.

[bkohrn]

I've been seeing a similar issue (in an incredibly complex pipeline which may not be appropriate to debugging; see #771), but this example you gave doesn't seem to reproduce the behavior for me when I actually run it. I don't think the dryrun is an appropriate test, since the order of jobs in the dryrun isn't necessarily the order in which jobs will actually be executed. Unfortunately, I'm having trouble replicating it in a simpler pipeline.

[kernco]

I see, well if it helps the pipeline that trimmed down to get the minimal example is the MERlin software for analyzing MERFISH data developed by Xiaowei Zhuang's lab: https://github.com/emanuega/MERlin