Skip to content

Commit

Permalink
feat: Unified seqtk wrapper (#1777)
Browse files Browse the repository at this point in the history
<!-- Ensure that the PR title follows conventional commit style (<type>:
<description>)-->
<!-- Possible types are here:
https://github.com/commitizen/conventional-commit-types/blob/master/index.json
-->

### Description

<!-- Add a description of your PR here-->

### QC
<!-- Make sure that you can tick the boxes below. -->

* [x] I confirm that:

For all wrappers added by this PR, 

* there is a test case which covers any introduced changes,
* `input:` and `output:` file paths in the resulting rule can be changed
arbitrarily,
* either the wrapper can only use a single core, or the example rule
contains a `threads: x` statement with `x` being a reasonable default,
* rule names in the test case are in
[snake_case](https://en.wikipedia.org/wiki/Snake_case) and somehow tell
what the rule is about or match the tools purpose or name (e.g.,
`map_reads` for a step that maps reads),
* all `environment.yaml` specifications follow [the respective best
practices](https://stackoverflow.com/a/64594513/2352071),
* wherever possible, command line arguments are inferred and set
automatically (e.g. based on file extensions in `input:` or `output:`),
* all fields of the example rules in the `Snakefile`s and their entries
are explained via comments (`input:`/`output:`/`params:` etc.),
* `stderr` and/or `stdout` are logged correctly (`log:`), depending on
the wrapped tool,
* temporary files are either written to a unique hidden folder in the
working directory, or (better) stored where the Python function
`tempfile.gettempdir()` points to (see
[here](https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir);
this also means that using any Python `tempfile` default behavior
works),
* the `meta.yaml` contains a link to the documentation of the respective
tool or command,
* `Snakefile`s pass the linting (`snakemake --lint`),
* `Snakefile`s are formatted with
[snakefmt](https://github.com/snakemake/snakefmt),
* Python wrapper scripts are formatted with
[black](https://black.readthedocs.io).
* Conda environments use a minimal amount of channels, in recommended
ordering. E.g. for bioconda, use (conda-forge, bioconda, nodefaults, as
conda-forge should have highest priority and defaults channels are
usually not needed because most packages are in conda-forge nowadays).
  • Loading branch information
fgvieira committed Oct 26, 2023
1 parent bb29891 commit 0e7fff5
Show file tree
Hide file tree
Showing 28 changed files with 182 additions and 234 deletions.
Expand Up @@ -4,3 +4,4 @@ channels:
- nodefaults
dependencies:
- seqtk =1.4
- pigz
7 changes: 0 additions & 7 deletions bio/seqtk/mergepe/environment.yaml

This file was deleted.

19 changes: 0 additions & 19 deletions bio/seqtk/mergepe/meta.yaml

This file was deleted.

13 changes: 0 additions & 13 deletions bio/seqtk/mergepe/test/Snakefile

This file was deleted.

16 changes: 0 additions & 16 deletions bio/seqtk/mergepe/wrapper.py

This file was deleted.

15 changes: 15 additions & 0 deletions bio/seqtk/meta.yaml
@@ -0,0 +1,15 @@
name: seqtk
description: Toolkit for processing sequences in FASTA/Q formats
url: https://github.com/lh3/seqtk
authors:
- Filipe G. Vieira
input:
- fastx file(s) (can be gzip bcompressed)
output:
- fastn files (can be gzip bcompressed)
params:
- n: number of reads after subsampling (for `sample`)
- extra: additional program options (e.g. `-s` for `sample` or `-b/-e` for `trimfq`)
- compress_lvl: compression level (see `gzip` manual for details)
notes: |
* Multiple threads can be used during compression of the output file.
10 changes: 0 additions & 10 deletions bio/seqtk/seq/meta.yaml

This file was deleted.

11 changes: 0 additions & 11 deletions bio/seqtk/seq/test/Snakefile

This file was deleted.

15 changes: 0 additions & 15 deletions bio/seqtk/seq/wrapper.py

This file was deleted.

7 changes: 0 additions & 7 deletions bio/seqtk/subsample/pe/environment.yaml

This file was deleted.

12 changes: 0 additions & 12 deletions bio/seqtk/subsample/pe/meta.yaml

This file was deleted.

16 changes: 0 additions & 16 deletions bio/seqtk/subsample/pe/test/Snakefile

This file was deleted.

Binary file removed bio/seqtk/subsample/pe/test/a.1.fastq.gz
Binary file not shown.
Binary file removed bio/seqtk/subsample/pe/test/a.2.fastq.gz
Binary file not shown.
31 changes: 0 additions & 31 deletions bio/seqtk/subsample/pe/wrapper.py

This file was deleted.

7 changes: 0 additions & 7 deletions bio/seqtk/subsample/se/environment.yaml

This file was deleted.

12 changes: 0 additions & 12 deletions bio/seqtk/subsample/se/meta.yaml

This file was deleted.

14 changes: 0 additions & 14 deletions bio/seqtk/subsample/se/test/Snakefile

This file was deleted.

Binary file removed bio/seqtk/subsample/se/test/a.fastq.gz
Binary file not shown.
24 changes: 0 additions & 24 deletions bio/seqtk/subsample/se/wrapper.py

This file was deleted.

91 changes: 91 additions & 0 deletions bio/seqtk/test/Snakefile
@@ -0,0 +1,91 @@
rule seqtk_seq_fq2fas:
input:
"reads/{prefix}.fastq",
output:
"results/fq2fas/{prefix}.fasta",
log:
"logs/fq2fas/{prefix}.log",
params:
command="seq",
extra="-A",
wrapper:
"master/bio/seqtk"


rule seqtk_seq_convBQ:
input:
"reads/{prefix}.fastq",
output:
"results/convBQ/{prefix}.fasta",
log:
"logs/convBQ/{prefix}.log",
params:
command="seq",
extra="-aQ 64 -q 20 -n N",
wrapper:
"master/bio/seqtk"


rule seqtk_subseq_list:
input:
"reads/{prefix}.fastq",
"reads/id.list",
output:
"results/subseq_list/{prefix}.fq.gz",
log:
"logs/subseq_list/{prefix}.log",
params:
command="subseq",
extra="",
wrapper:
"master/bio/seqtk"


rule seqtk_mergepe:
input:
r1="reads/{sample}.1.fastq.gz",
r2="reads/{sample}.2.fastq.gz",
output:
merged="results/mergepe/{sample}.fastq.gz",
log:
"logs/mergepe/{sample}.log",
params:
command="mergepe",
compress_lvl=9,
threads: 2
wrapper:
"master/bio/seqtk"


rule seqtk_sample_se:
input:
"reads/{sample}.fastq.gz",
output:
"results/sample_se/{sample}.fastq.gz",
log:
"logs/sample_se/{sample}.log",
params:
command="sample",
n=3,
extra="-s 12345",
threads: 1
wrapper:
"master/bio/seqtk"


rule seqtk_sample_pe:
input:
f1="reads/{sample}.1.fastq.gz",
f2="reads/{sample}.2.fastq.gz",
output:
f1="results/sample_pe/{sample}.1.fastq.gz",
f2="results/sample_pe/{sample}.2.fastq.gz",
log:
"logs/sample_pe/{sample}.log",
params:
command="sample",
n=3,
extra="-s 12345",
threads: 1
wrapper:
"master/bio/seqtk"
File renamed without changes.
File renamed without changes.
File renamed without changes.
Binary file added bio/seqtk/test/reads/a.fastq.gz
Binary file not shown.
1 change: 1 addition & 0 deletions bio/seqtk/test/reads/id.list
@@ -0,0 +1 @@
1
31 changes: 31 additions & 0 deletions bio/seqtk/wrapper.py
@@ -0,0 +1,31 @@
"""Snakemake wrapper for SeqTk."""

__author__ = "Filipe G. Vieira"
__copyright__ = "Copyright 2023, Filipe G. Vieira"
__license__ = "MIT"

from snakemake.shell import shell

log = snakemake.log_fmt_shell(stdout=False, stderr=True, append=False)
extra = snakemake.params.get("extra", "")
compress_lvl = snakemake.params.get("compress_lvl", "6")

pipe_comp = (
f"| pigz --processes {snakemake.threads} -{compress_lvl} --stdout"
if snakemake.output[0].endswith(".gz")
else ""
)

if snakemake.params.command == "sample":
n_reads = snakemake.params.get("n", "")
assert len(snakemake.input) == len(
snakemake.output
), "Command 'sample' requires same number of input and output files."
for in_fx, out_fx in zip(snakemake.input, snakemake.output):
shell(
"(seqtk {snakemake.params.command} {extra} {in_fx} {n_reads} {pipe_comp} > {out_fx}) {log}"
)
else:
shell(
"(seqtk {snakemake.params.command} {extra} {snakemake.input} {pipe_comp} > {snakemake.output}) {log}"
)

0 comments on commit 0e7fff5

Please sign in to comment.