Python error when creating subdirectories #1593
Comments
|
I have a very similar issue here with the following traceback : Traceback (most recent call last):
File "/dev/envs/shared/pipeline/lib/python3.10/site-packages/snakemake/sourcecache.py", line 330, in __init__
os.makedirs(self.cache, exist_ok=True)
File "/dev/envs/shared/pipeline/lib/python3.10/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/dev/envs/shared/pipeline/lib/python3.10/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/dev/envs/shared/pipeline/lib/python3.10/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
[Previous line repeated 2 more times]
File "/dev/envs/shared/pipeline/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/home'I added some error handling to the line in sourcecache.py, to see which folder was trying to be created exactly, and this is the one that fails : File "/labos/UGM/dev/envs/shared/pipeline/lib/python3.10/site-packages/snakemake/sourcecache.py", line 332, in __init__
raise Exception(f"{os.environ.get('USER')} could not create folder tree : '{Path(self.cache).resolve().absolute()}'")
Exception: user could not create folder tree : '/home/user/.cache/snakemake/snakemake/source-cache'This folder actually exists, so this is really mind-boggling and I'm running out of ideas to debug this issue. |
|
I ran into the same problem today running I found the environment variable, snakemake/snakemake/sourcecache.py Line 327 in 2b6950c I managed to fix this issue by setting: which is used by https://github.com/ActiveState/appdirs to set the cache path. I think the documentation for |
|
I'm pretty sure the SNAKEMAKE_OUTPUT_CACHE has nothing to do with the runtime cache where the error is raised, as it is used to cache common results from rules between workflows. @james-s-willis |
|
I managed to fix the issue with something that looks completely unrelated. |
|
@LukaP-BB thanks for the tip, I didn't realise that Regarding your fix, I can see how setting |
|
I had the same issue and I solved by modifying the container. I added this rule to dockerfile:
|
|
Thanks to @moxgreen -- this solution worked great! |
|
I had the same problem running snakemake 7.3.8 (installed via pip) on my slurm cluster. On my cluster, HOME directory is not modifiable on the compute nodes. I solved the problem by manually setting the self.cache in sourcecache.py to my scratch directory. |
resolves #2031 resolves #1593 ### QC <!-- Make sure that you can tick the boxes below. --> * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).
|
Wow, thank you @peter-yufan-zeng and @moxgreen! I tried de-installing and re-installing and updating snakemake, but kept getting the same error. Finally I found your post and I manually set the self.cache in |
Snakemake version
7.3.8
Describe the bug
This may be related to #1588
I need to run Snakemake within a Docker container. Workflows that used snakemake 5.29.0 work fine.
When I updated to the latest version (7.3.8), there is an error when creating temporary directories.
I'll try to narrow it down further tomorrow but the error appeared between 6.8 and 6.11.1The issue appears to arise between 6.8.0 and 6.8.1.
Logs
Additional context
Running in a Docker container within an HTCondor cluster.
The text was updated successfully, but these errors were encountered: