Skip to content

Commit

Permalink
feat: add memory inference to samtools sort (#1208)
Browse files Browse the repository at this point in the history
<!-- Ensure that the PR title follows conventional commit style (<type>:
<description>)-->
<!-- Possible types are here:
https://github.com/commitizen/conventional-commit-types/blob/master/index.json
-->

### Description

<!-- Add a description of your PR here-->
Add memory inference to Samtools Sort

### QC
<!-- Make sure that you can tick the boxes below. -->

* [x] I confirm that:

For all wrappers added by this PR, 

* there is a test case which covers any introduced changes,
* `input:` and `output:` file paths in the resulting rule can be changed
arbitrarily,
* either the wrapper can only use a single core, or the example rule
contains a `threads: x` statement with `x` being a reasonable default,
* rule names in the test case are in
[snake_case](https://en.wikipedia.org/wiki/Snake_case) and somehow tell
what the rule is about or match the tools purpose or name (e.g.,
`map_reads` for a step that maps reads),
* all `environment.yaml` specifications follow [the respective best
practices](https://stackoverflow.com/a/64594513/2352071),
* wherever possible, command line arguments are inferred and set
automatically (e.g. based on file extensions in `input:` or `output:`),
* all fields of the example rules in the `Snakefile`s and their entries
are explained via comments (`input:`/`output:`/`params:` etc.),
* `stderr` and/or `stdout` are logged correctly (`log:`), depending on
the wrapped tool,
* temporary files are either written to a unique hidden folder in the
working directory, or (better) stored where the Python function
`tempfile.gettempdir()` points to (see
[here](https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir);
this also means that using any Python `tempfile` default behavior
works),
* the `meta.yaml` contains a link to the documentation of the respective
tool or command,
* `Snakefile`s pass the linting (`snakemake --lint`),
* `Snakefile`s are formatted with
[snakefmt](https://github.com/snakemake/snakefmt),
* Python wrapper scripts are formatted with
[black](https://black.readthedocs.io).
* Conda environments use a minimal amount of channels, in recommended
ordering. E.g. for bioconda, use (conda-forge, bioconda, nodefaults, as
conda-forge should have highest priority and defaults channels are
usually not needed because most packages are in conda-forge nowadays).
  • Loading branch information
fgvieira committed May 12, 2023
1 parent d613ddf commit d70a806
Show file tree
Hide file tree
Showing 6 changed files with 23 additions and 12 deletions.
2 changes: 1 addition & 1 deletion bio/samtools/sort/environment.yaml
Expand Up @@ -4,4 +4,4 @@ channels:
- nodefaults
dependencies:
- samtools =1.16.1
- snakemake-wrapper-utils =0.5.2
- snakemake-wrapper-utils =0.5.3
11 changes: 8 additions & 3 deletions bio/samtools/sort/meta.yaml
@@ -1,8 +1,13 @@
name: samtools sort
description: Sort bam file with samtools.
url: http://www.htslib.org/doc/samtools-sort.html
authors:
- Johannes Köster
- Filipe G. Vieira
notes: |
* The `extra` param allows for additional program arguments (not `-@/--threads`, `--write-index`, `-o` or `-O/--output-fmt`).
* For more information see, http://www.htslib.org/doc/samtools-sort.html
input:
- SAM/BAM/CRAM file
output:
- SAM/BAM/CRAM file
- SAM/BAM/CRAM index file (optional)
params:
- extra: additional program arguments (not `-@/--threads`, `--write-index`, `-m`, `-o` or `-O/--output-fmt`).
6 changes: 4 additions & 2 deletions bio/samtools/sort/wrapper.py
Expand Up @@ -7,17 +7,19 @@
import tempfile
from pathlib import Path
from snakemake.shell import shell
from snakemake_wrapper_utils.snakemake import get_mem
from snakemake_wrapper_utils.samtools import get_samtools_opts


samtools_opts = get_samtools_opts(snakemake)
extra = snakemake.params.get("extra", "")
log = snakemake.log_fmt_shell(stdout=True, stderr=True)

mem_per_thread_mb = int(get_mem(snakemake) / snakemake.threads)

with tempfile.TemporaryDirectory() as tmpdir:
tmp_prefix = Path(tmpdir) / "samtools_fastq.sort_"
tmp_prefix = Path(tmpdir) / "samtools_sort"

shell(
"samtools sort {samtools_opts} {extra} -T {tmp_prefix} {snakemake.input[0]} {log}"
"samtools sort {samtools_opts} -m {mem_per_thread_mb}M {extra} -T {tmp_prefix} {snakemake.input[0]} {log}"
)
10 changes: 7 additions & 3 deletions bio/samtools/stats/meta.yaml
@@ -1,8 +1,12 @@
name: samtools stats
description: Generate stats using samtools.
url: http://www.htslib.org/doc/samtools-stats.html
authors:
- Julian de Ruiter
- Filipe G. Vieira
notes: |
* The `extra` param allows for additional program arguments (not `-@/--threads`).
* For more information see, http://www.htslib.org/doc/samtools-stats.html
input:
- SAM/BAM/CRAM file
output:
- statistics file
params:
- extra: additional program arguments (not `-@/--threads`).
2 changes: 1 addition & 1 deletion bio/samtools/stats/test/Snakefile
@@ -1,7 +1,7 @@
rule samtools_stats:
input:
bam="mapped/{sample}.bam",
bed="design.bed", #Optional input, specify target regions
bed="design.bed", #Optional input, specify target regions
output:
"samtools_stats/{sample}.txt",
params:
Expand Down
4 changes: 2 additions & 2 deletions bio/samtools/stats/wrapper.py
Expand Up @@ -12,7 +12,7 @@

bed = snakemake.input.get("bed", "")
if bed:
bed = "-t " + bed
bed = f"-t {bed}"

samtools_opts = get_samtools_opts(
snakemake, parse_write_index=False, parse_output=False, parse_output_format=False
Expand All @@ -25,5 +25,5 @@


shell(
"samtools stats {samtools_opts} {extra} {snakemake.input.bam} {bed} {region} > {snakemake.output} {log}"
"samtools stats {samtools_opts} {extra} {snakemake.input[0]} {bed} {region} > {snakemake.output[0]} {log}"
)

0 comments on commit d70a806

Please sign in to comment.