verkko stops with no error just after buildPackages

[brianwalenz]

Verkko stops early without generating consensus sequences.

checkpoint buildPackages:
    input: [...]/ecoli_hifi_subset24x.fastq.gz, 6-layoutContigs/unitig-popped.layout, 7-consensus/ont_subset.fasta.gz
    output: 7-consensus/packages.tigName_to_ID.map, 7-consensus/packages.finished
    log: 7-consensus/buildPackages.err
    jobid: 13
    reason: Missing output files: 7-consensus/packages.tigName_to_ID.map, 7-consensus/packages.finished; Input files updated by another job: 6-layoutContigs/unitig-popped.layout, 7-consensus/ont_subset.fasta.gz
    threads: 8
    resources: tmpdir=/tmp, job_id=1, n_cpus=8, mem_gb=120, time_h=24
Downstream jobs will be updated after completion.

Submitted job 13 with external jobid '299522'.
[Tue Apr 19 14:42:21 2022]
Finished job 13.
14 of 16 steps (88%) done
Updating job combineConsensus.
Complete log: [...]/.snakemake/log/2022-04-19T143841.803652.snakemake.log

This might be related to snakemake/snakemake#823.

It affects Snakemake 6.10.0, but not 7.3.7.

It seems to affect only grid-based runs (verkko option --sge or --slurm).

It does NOT stop early if the job group defined in 6-layoutContigs.sm is removed.

If verkko is simply restarted after the early stop, it will run to completion.

rule generateLayoutContigsInputs:
    [...]
    group:
        'layoutGroup'

rule layoutContigs:
    [...]
    group:
        'layoutGroup'

[skoren]

Solution is to update to latest snakemake