Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Patching samtools sort process behaviour when samtools sort resorts to using on-disk temporary files #1222

Closed
wants to merge 2 commits into from
Closed

Patching samtools sort process behaviour when samtools sort resorts to using on-disk temporary files #1222

wants to merge 2 commits into from

Conversation

koustav-pal
Copy link

PR checklist

  • [ x] This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/rnaseq branch on the nf-core/test-datasets repository.
  • [ x] Make sure your code lints (nf-core lint).
  • [x ] Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • [ x] Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

samtools sort resorts to using a combination of on-disk and in-memory
temporary files when it is unable to sort using memory only. In such cases,
samtools issues an exit code of 140 and provides a suitable error message
stating that it is using in-memory and on-disk temporary files.
This patch modulates samtools sort errorStrategy when such a scenario is
observed.

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 23, 2024 •
edited

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit f2a99c1

+| ✅ 170 tests passed       |+
#| ❔   7 tests were ignored |#
!| ❗   7 tests had warnings |!

❗ Test warnings:

  • files_exist - File not found: assets/multiqc_config.yml
  • files_exist - File not found: .github/workflows/awstest.yml
  • files_exist - File not found: .github/workflows/awsfulltest.yml
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!

❔ Tests ignored:

  • files_exist - File is ignored: conf/modules.config
  • nextflow_config - Config default ignored: params.ribo_database_manifest
  • files_unchanged - File ignored due to lint config: assets/email_template.html
  • files_unchanged - File ignored due to lint config: assets/email_template.txt
  • files_unchanged - File ignored due to lint config: .gitignore or .prettierignore or pyproject.toml
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/rnaseq/rnaseq/.github/workflows/awstest.yml
  • multiqc_config - 'assets/multiqc_config.yml' not found

✅ Tests passed:

Run details

  • nf-core/tools version 2.13
  • Run at 2024-02-23 15:15:44

@drpatelh
Copy link
Member

Hi @koustav-pal ! Thanks for the PR! Can you provide me an example of the stdout/ Nextflow log file when there is a failure here please? Is it a silent error?

In any case, this PR should be made to nf-core/modules to update this module there. It can then be installed in pipelines from there.

We also should be catching this exit status with the standard errorStrategy we have set across all processes:

rnaseq/conf/base.config

Line 17 in b89fac3

errorStrategy = { task.exitStatus in ((130..145) + 104) ? 'retry' : 'finish' }

@drpatelh drpatelh added this to the 3.15.0 milestone May 13, 2024
@drpatelh
Copy link
Member

Closing for now. Please feel free to re-open with a fix as suggested above. Be great to have a little more info on the failure and access to the logs generating it.

@drpatelh drpatelh closed this May 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@drpatelh drpatelh Awaiting requested review from drpatelh

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
3.15.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@koustav-pal @drpatelh