Incompatibility between Snakemake (v7.0.0 or later) and Tibanna?

[dvav]

Snakemake version: 7.1.1

Describe the bug

I 've been using Snakemake with Tibanna for some time now without any problem. However, it seems that I cannot make Snakemake v7.0.0 or higher work with Tibanna (see traceback below). It works fine, though, with Snakemake v6.15.5. Any idea what's wrong? Maybe it has something to do with Snakemake v7.0.0 requiring at least Python v3.7?

Logs

Traceback (most recent call last):
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/utils.py", line 430, in format
    return fmt.format(_pattern, *args, **variables)
  File "/usr/lib/python3.9/string.py", line 161, in format
    return self.vformat(format_string, args, kwargs)
  File "/usr/lib/python3.9/string.py", line 165, in vformat
    result, _ = self._vformat(format_string, args, kwargs, used_args, 2)
  File "/usr/lib/python3.9/string.py", line 205, in _vformat
    obj, arg_used = self.get_field(field_name, args, kwargs)
  File "/usr/lib/python3.9/string.py", line 270, in get_field
    obj = self.get_value(first, args, kwargs)
  File "/usr/lib/python3.9/string.py", line 227, in get_value
    return kwargs[key]
KeyError: 'job_specific_args'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/__init__.py", line 714, in snakemake
    success = workflow.execute(
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/workflow.py", line 1097, in execute
    success = self.scheduler.schedule()
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/scheduler.py", line 540, in schedule
    self.run(runjobs)
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/scheduler.py", line 588, in run
    executor.run_jobs(
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 153, in run_jobs
    self.run(
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 2360, in run
    tibanna_input = self.make_tibanna_input(job)
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 2344, in make_tibanna_input
    self.add_command(job, tibanna_args, tibanna_config)
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 2221, in add_command
    command = self.format_job_pattern(
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/executors/__init__.py", line 375, in format_job_pattern
    cmd = format(
  File "/home/dimitris/Tmp/test_smk/venv/lib/python3.9/site-packages/snakemake/utils.py", line 432, in format
    if str(ex).strip("'") in variables["wildcards"].keys():
KeyError: 'wildcards'

Minimal example

Snakefile (under workflow/):

ID = ['SAMPLE1', 'SAMPLE2']

rule all:
    input:
        tag = expand('results/TEST/{id}/{id}.txt', id = ID)

rule my_rule:
    output:
        tag = 'results/TEST/{id}/{id}.txt'
    shell: "touch {output.tag}"

Profile config.yaml (under workflow/config/aws/):

tibanna: true
tibanna-sfn: tibanna_unicorn_something
default-remote-prefix: mybucket/sudir/
default-resources:
  - disk_mb=8000
tibanna-config:
  - instance_type=t2.micro
  - log_bucket=mylogbucket
keep-going: true

Run as: snakemake --jobs 2 --profile wokflow/config/aws/ -n (works fine)
Run as: snakemake --jobs 2 --profile wokflow/config/aws/ (see traceback above)

Additional context None

[SooLee]

It looks like some new incompatibility issue. I can take a look this weekend.

[hwalinga]

Seems the call in the tibanna executor should be format_job and not format_job_pattern.

[dvav]

OK, many thanks for sorting this out @SooLee , @hwalinga and @johanneskoester. Looking forward to the updated version of the software.