QUALIMAP_RNASEQ error simultaneous access on same dir #1298
Comments
drpatelh
added
question
|
I think there is progress if I restart the process over and over again. But that takes days to complete for a set of samples. Can anyone tell me what is the actual problem here? |
|
@redvoidling I think I can see a possible solution here. In your local copy of the rnaseq workflow, could you try a fix for me? In modules/nf-core/qualimap/rnaseq/main.nf, change: to: If that does the trick I'll PR the fix in the usual way. |
|
Thank you! I will restart and get back to you on that. Another thought of mine would be to just run the Qualimap process in a loop but I do not think there is a feature to just execute a subprocess with the usual nextflow commands. |
I'm not sure what you mean. You could limit the number of parallel qualimap tasks with maxForks, which might work, but we should fix it 'properly'. |
same error! I was under the impression that creating an individual tmp folder for each sample would solve the issue, but that is not what the snipped you suggested does right? |
|
That seems to have done the trick in removing the blocking error. My additional snippet just disabled some logging that probably isn't essential and seemed to be clashing. But it doesn't seem to have helped the run time: You may just need to increase the runtime limit for that process. |
|
I have not seen the blocking in every error log I just assumed it to be the root cause because 8 hours seems to be quite extensive for Qualimap to run on one single sample. How would I go about increasing the runtime limit? to because that would give it 16h of compute time. Hope it works. |
|
The log suggests to me that it's the writing of results that is taking time, which may be related to your local disks or other infrastructure. You can supply custom configuration to override resource configuration for specific processes- see the instructions on nf-core: https://nf-co.re/docs/usage/configuration#tuning-workflow-resources |
|
Ok I think changing the label did the trick because, like I said, it increased the runtime to 16h. Thank you for your help! :) |
|
Glad it's working. Just be aware that changing the label changes more than just the time request, so you may for example be requesting more memory than you need. |
Description of the bug
The problem seems to be that more than one process tries to access the same tmp dir:
Command used and terminal output
Relevant files
No response
System information
Nextflow: version 23.10.1 build 5891
HPC
slurm
LINUX, ROCKY 8.9
Container engine: Apptainer
rnaseq: 3.14.0
The text was updated successfully, but these errors were encountered: