New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Job failure with shadow directive if log files not created #1257
Labels
bug
Something isn't working
Comments
jaicher
added a commit
to jaicher/snakemake
that referenced
this issue
Nov 12, 2021
jaicher
added a commit
to jaicher/snakemake
that referenced
this issue
Nov 12, 2021
This addresses gh snakemake#1257. If the rule did not create a logfile, an empty one would be created in its final destination. If the shadow directive was used, however, Snakemake would try to move the nonexistent logfile from the shadow directory to its final location. This caused the observed failures. This commit has Snakemake create the empty file where Snakemake would have expected the program to create it (i.e. in the shadow directory). This enables the unshadow operation that takes place later to succeed.
2 tasks
johanneskoester
added a commit
that referenced
this issue
Nov 27, 2021
#1258) * Test for issue #1257 fails for missing logfile with shadow directory * If rule did not create logfile, create it in its shadow directory This addresses gh #1257. If the rule did not create a logfile, an empty one would be created in its final destination. If the shadow directive was used, however, Snakemake would try to move the nonexistent logfile from the shadow directory to its final location. This caused the observed failures. This commit has Snakemake create the empty file where Snakemake would have expected the program to create it (i.e. in the shadow directory). This enables the unshadow operation that takes place later to succeed. * move remote handling of log files into the general remote output handling; fix existence check Co-authored-by: Johannes Köster <johannes.koester@tu-dortmund.de> Co-authored-by: Johannes Köster <johannes.koester@uni-due.de>
Fixed with PR #1258. |
maarten-k
added a commit
to maarten-k/realignment
that referenced
this issue
May 18, 2022
…og were "file not found")
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Snakemake version: v6.10.0
Describe the bug
For any job that specifies a logfile: if I am not using the shadow directive, it is okay if my job fails to make a logfile itself (because Snakemake creates an empty one at the final location for me, it looks like). But, if I use the shadow directive, I must explicitly make sure that a logfile is created in the specified location, otherwise, my job fails with
FileNotFoundError
insnakemake.dag.unshadow_output(...)
because the logfile does not exist in the shadow directory, so it cannot move it to its final desired location. I think that if it succeeds without the shadow directive, it should succeed with the shadow directive.Minimal example
As is, this fails. from
FileNotFoundError
inunshadow_output(...)
. Comment out the shadow directive, and it works.This seems like it'd be a pretty easy fix: either automatically touch the logfile in the shadow directory rather than the original location, or ignore FileNotFoundError for logfiles.
The text was updated successfully, but these errors were encountered: