Sniffles v2.3.2 never exits

[AlexanRNA]

Hello!

Thank you very much for your great tool! Last week I have been trying multisample SV calling with the new sniffles2 2.3.2 version, but I keep running into an issue. Sniffles runs normally, but once the output VCF file + index is written, it never terminates, it doesn't exit normally. The output file looks fine, but the process is only terminated once my job runs out of time on our HPC cluster.

I tried to run sniffles2 v2.3.2 both from conda and from docker container I made (alexanrna/sniffles:v2.3.2), leading to same issue. I do not have this issue when running the same command with the 2.2 version.

Command I run :

sniffles --input input.tsv \
--threads 24 --vcf /path/to/vcf/${current_date}_multisample_sniffles_2.3.2_conda.vcf.gz

The last few lines of log file (there is no error message though):

2024-04-19 17:27:12,154 INFO sniffles.worker (215319): Worker 18 got result for task #302
2024-04-19 17:27:12,154 INFO sniffles.worker (215319): Worker 18 shutting down...
2024-04-19 17:27:12,686 INFO sniffles.worker (215319): Worker 18 done
2024-04-19 17:27:18,419 INFO sniffles.worker (215319): Worker 11 got result for task #301
2024-04-19 17:27:18,419 INFO sniffles.worker (215319): Worker 11 shutting down...
2024-04-19 17:27:18,944 INFO sniffles.worker (215319): Worker 11 done
2024-04-19 17:29:03,889 INFO sniffles.worker (215319): Worker 6 got result for task #303
2024-04-19 17:29:03,889 INFO sniffles.worker (215319): Worker 6 shutting down...
2024-04-19 17:29:13,900 INFO sniffles.worker (215319): Worker 6 done
2024-04-19 17:29:13,941 INFO sniffles.main (215319): Took 19459.51s.
2024-04-19 17:29:13,941 INFO sniffles.main (215319): 
2024-04-19 17:30:28,397 INFO sniffles.main (215319): Generating index for /path/to/vcf/20240419_multisample_sniffles_2.3.2_conda.vcf.gz...
2024-04-19 17:30:35,951 INFO sniffles.main (215319): Indexing VCF output took 7.55s.
2024-04-19 17:30:35,951 INFO sniffles.main (215319): Wrote 207384 called SVs to /path/to/vcf/20240419_multisample_sniffles_2.3.2_conda.vcf.gz (multi-sample, sorted, bgzipped, tabix-indexed)

Do you know what could be going on and how to solve this?

Thanks a lot!

Alex

[fellen31]

I see the same problem & only with multisample calling.

[tnguyengel]

I also get this issue with multisample calling. It doesn't happen to all my combination of multi-sample inputs, but when it does, it happens consistently. I can't share the samples that produce this issue. However, are there debug logs that I can upload to help the developers debug? If so, how would I go about producing them?

[hermannromanek]

Hi @tnguyengel

Yes, the logs would help. If possible, could you upload it somewhere, e.g. to google drive?

Thanks,
Hermann

[hermannromanek]

Thanks for reporting @AlexanRNA - can you upload the whole log someplace? Or a few more lines, it looks like the worker processes don't shut down properly.

Thanks,
Hermann

[AlexanRNA]

Sure! the full log, names and paths were changed:
sniffles_log.txt

Let me know if I can upload anything else to help!

Cheers,

Alex

[tnguyengel]

Yes, the logs would help. If possible, could you upload it somewhere, e.g. to google drive?

I'm uploading default stdout logs to github:

1714058589.629290.out.txt

[wdyyy]

I found that setting the number of threads to 1 makes the program exit normally, so I think the issue may lie in the multithreading aspect.

[hermannromanek]

Thank you for the logs @AlexanRNA @tnguyengel

Both of them confirm its the same bug - a worker process refusing to shut down gracefully.

While initially I was not able to reproduce it on my system, it did reproduce a few times while using Alex' docker container - thank you very much for sharing that @AlexanRNA .

I just pushed a quick fix to have sniffles exit properly, although I'm afraid the real cause will require some more investigation and (if I can confirm my suspected cause) a more invasive code change. @wdyyy your assumption is close - its a timing issue - I believe this bug may also present with only one thread (albeit with much lower probability), and a higher number of threads makes it much more likely to occur.

Thanks,
Hermann

[alvizain786]

A quick note, I experienced this issue with the conda version as well. I hope this gets fixed with a future release of the tool. SV analysis could take a long time with larger genomes, i.e. human.

[AlexanRNA]

Hello, I tried the new version, 2.3.3 and it exits properly now. Thanks a lot for a quick fix!

[hermannromanek]

Thanks for testing :)