Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filenames split at whitespaces when using --cluster #1066

Open
fneum opened this issue Jun 29, 2021 · 2 comments
Open

Filenames split at whitespaces when using --cluster #1066

fneum opened this issue Jun 29, 2021 · 2 comments
Labels
bug Something isn't working

Comments

@fneum
Copy link
Contributor

fneum commented Jun 29, 2021

Snakemake version

6.5.1

(did not yet track down the version where this first occurred, but somewhere after 6.1.0)

Describe the bug

Only when using snakemake --cluster ..., input filenames are split at whitespaces resulting in missing files.

Possibly related to #1007?

Logs

Waiting at most 60 seconds for missing files.
Missing files after 60 seconds:
data/biomass/JRC
Biomass
Potentials.xlsx

Minimal example

rule build_biomass_potentials:
    input:
        jrc_potentials="data/biomass/JRC Biomass Potentials.xlsx"
    output:
        biomass_potentials="resources/biomass_potentials.csv"
    script:
        "scripts/build_biomass_potentials.py"

Additional context

@fneum fneum added the bug Something isn't working label Jun 29, 2021
@G-kodes
Copy link

G-kodes commented Jul 16, 2021

I can confirm this seems to be happening on my side as well when using an input function designed to return a path of the form: /storage/users/user/Reference Genomes/GRCh38.fa.gz, I end up getting a debug message stating :

Missing files after 5 seconds:
/storage/users/user/Reference
Genomes/GRCh38.fa.gz

I have tracked it back to this stack overflow submission, where it was pointed out by @Manavalan Gajapathy that you can use quotation marks in shell scripts to honour the whitespace in the command itself. With that in mind, this behaviour would indicate that the fault lies in some kind of check-up script that is run by snakemake to 'watch' for the output files?

@G-kodes
Copy link

G-kodes commented Jul 16, 2021

Just a quick update. I changed my filename from Reference Genome to Reference_Genome and viola! Worked like a bomb. Definitely seems like something to do with how Snakemake handles file waiting.

As a side note, I did try to dig around the codebase myself, but despite getting pretty far along, I cant seem to find any functions relating to the wait_for_files group of functions called on inputs spesifically, much fewer inputs from function-based inputs as is my case (Though between @fneum and I's issue, it would indicate it's not an issue with function-based inputs specifically, just inputs in general?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants