Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Environment variables are not correctly quoted when executing job on cluster #860

Closed
kpj opened this issue Jan 30, 2021 · 3 comments · Fixed by #1491
Closed

Environment variables are not correctly quoted when executing job on cluster #860

kpj opened this issue Jan 30, 2021 · 3 comments · Fixed by #1491
Labels
bug Something isn't working

Comments

@kpj
Copy link
Contributor

kpj commented Jan 30, 2021

Snakemake version
v5.32.0

Describe the bug
When using the envvars directive, submitting jobs on a cluster will fail with [list of env. vars] is not a command.

Minimal example
Consider this Snakefile:

envvars:
    'VARIABLE01',
    'VARIABLE02'

rule foo:
    output:
        fname='foo.csv'
    script:
        'foo.py'

This will generate a job script of this form

#!/bin/sh
# properties = {"type": "single", "rule": "foo" [...]
'VARIABLE01=XXX VARIABLE02=XXX' [...]

As you can see, the environment variables are quoted as a group which makes bash crash.

Additional context
The issue seems to be the usage of AlwaysQuotedFormatter in snakemake.utils when calling format for the job script with _quote_all=True. Changing this to _quote_all=False fixed the issue for me, but I am not sure if this breaks other cases.

@kpj kpj added the bug Something isn't working label Jan 30, 2021
@smutch
Copy link

smutch commented Mar 16, 2022

This seems to be loosely connected to #1007 in that it also involves quoting of environment variables (see also #1008). Rather than disabling _quote_all entirely I have circumvented this by manually prepending envvars to exec_job rather than letting format_job() handle it in

def write_jobscript(self, job, jobscript, **kwargs):

I'd be happy to provide a PR if I can get some guidance as to if this would be a satisfactory permanent fix.

@johanneskoester
Copy link
Contributor

First of all, sorry for the late response. After many new features, Snakemake is approaching feature completeness (at least for my personal agenda and at the moment). Hence, now is the time to fix or close all the open bugs.

Above PR provides a fix on a kind of global scale. Thanks a lot for pointing to the issue!

@smutch
Copy link

smutch commented Mar 16, 2022

No need to apologise! Thanks for the fix. I've just started working with Snakemake and have been finding it great so far!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants