feat: CNV Facets

[tdayris]

Description

This PR adds cnv_facets to the list of available wrappers

QC

• I confirm that:

For all wrappers added by this PR,

• there is a test case which covers any introduced changes,
• input: and output: file paths in the resulting rule can be changed arbitrarily,
• either the wrapper can only use a single core, or the example rule contains a threads: x statement with x being a reasonable default,
• rule names in the test case are in snake_case and somehow tell what the rule is about or match the tools purpose or name (e.g., map_reads for a step that maps reads),
• all environment.yaml specifications follow the respective best practices,
• wherever possible, command line arguments are inferred and set automatically (e.g. based on file extensions in input: or output:),
• all fields of the example rules in the Snakefiles and their entries are explained via comments (input:/output:/params: etc.),
• stderr and/or stdout are logged correctly (log:), depending on the wrapped tool,
• temporary files are either written to a unique hidden folder in the working directory, or (better) stored where the Python function tempfile.gettempdir() points to (see here; this also means that using any Python tempfile default behavior works),
• the meta.yaml contains a link to the documentation of the respective tool or command,
• Snakefiles pass the linting (snakemake --lint),
• Snakefiles are formatted with snakefmt,
• Python wrapper scripts are formatted with black.
• Conda environments use a minimal amount of channels, in recommended ordering. E.g. for bioconda, use (conda-forge, bioconda, nodefaults, as conda-forge should have highest priority and defaults channels are usually not needed because most packages are in conda-forge nowadays).

[tdayris]

Tests are passing on my side. I don't know how to reproduce this error that raises on Github.

My side:

(test-snakemake-wrappers) tdayris@latitude7400:~/Projects/snakemake-wrappers$ pytest test.py -vv -k cnv_face
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.11.5, pytest-7.4.0, pluggy-1.3.0 -- /home/tdayris/mambaforge/envs/test-snakemake-wrappers/bin/python3.11
cachedir: .pytest_cache
rootdir: /home/tdayris/Projects/snakemake-wrappers
collected 475 items / 474 deselected / 1 selected                                                                                                                                                          

test.py::test_cnv_facets XPASS                                                                                                                                                                       [100%]

============================================================================== 474 deselected, 1 xpassed in 115.33s (0:01:55) ==============================================================================

I'm using Ubuntu 22.04, python 3.11 and the environment test-snakemake-wrappers is up to date (through mamba update -n test-snakemake-wrappers --all )

Current error on Github is :

Traceback (most recent call last):
2023-08-30T12:08:03.9172993Z   File "/tmp/tmp06v6yc3k/test/.snakemake/scripts/tmp2v8xiome.wrapper.py", line 3, in <module>
2023-08-30T12:08:03.9181327Z     import sys; sys.path.extend(['/usr/share/miniconda3/envs/snakemake/lib/python3.11/site-packages', '/home/runner/.cache/snakemake/snakemake/source-cache/runtime-cache/tmp94nylpu5/file/tmp/tmp06v6yc3k/master/bio/cnv_facets', '/tmp/tmp06v6yc3k/master/bio/cnv_facets']); import pickle; snakemake = pickle.loads(b'\x80\x04\x95\xc2\x04\x00\x00\x00\x00\x00\x00\x8c\x10snakemake.script\x94\x8c\tSnakemake\x94\x93\x94)\x81\x94}\x94(\x8c\x05input\x94\x8c\x0csnakemake.io\x94\x8c\nInputFiles\x94\x93\x94)\x81\x94(\x8c\x0cT.sample.bam\x94\x8c\x0cN.sample.bam\x94\x8c\x14common.sample.vcf.gz\x94e}\x94(\x8c\x06_names\x94}\x94(\x8c\x05tumor\x94K\x00N\x86\x94\x8c\x06normal\x94K\x01N\x86\x94\x8c\x03vcf\x94K\x02N\x86\x94u\x8c\x12_allowed_overrides\x94]\x94(\x8c\x05index\x94\x8c\x04sort\x94eh\x18\x8c\tfunctools\x94\x8c\x07partial\x94\x93\x94h\x06\x8c\x19Namedlist._used_attribute\x94\x93\x94\x85\x94R\x94(h\x1e)}\x94\x8c\x05_name\x94h\x18sNt\x94bh\x19h\x1ch\x1e\x85\x94R\x94(h\x1e)}\x94h"h\x19sNt\x94bh\x10h\nh\x12h\x0bh\x14h\x0cub\x8c\x06output\x94h\x06\x8c\x0bOutputFiles\x94\x93\x94)\x81\x94(\x8c\x0eCNV_bam.vcf.gz\x94\x8c\x12genome_bam.cnv.png\x94\x8c\x10cnv_bam.hist.pdf\x94\x8c\x11qc_bam.spider.pdf\x94e}\x94(h\x0e}\x94(h\x14K\x00N\x86\x94\x8c\x03cnv\x94K\x01N\x86\x94\x8c\x04hist\x94K\x02N\x86\x94\x8c\x06spider\x94K\x03N\x86\x94uh\x16]\x94(h\x18h\x19eh\x18h\x1ch\x1e\x85\x94R\x94(h\x1e)}\x94h"h\x18sNt\x94bh\x19h\x1ch\x1e\x85\x94R\x94(h\x1e)}\x94h"h\x19sNt\x94bh\x14h,h3h-h5h.h7h/ub\x8c\x06params\x94h\x06\x8c\x06Params\x94\x93\x94)\x81\x94\x8c\x00\x94a}\x94(h\x0e}\x94\x8c\x05extra\x94K\x00N\x86\x94sh\x16]\x94(h\x18h\x19eh\x18h\x1ch\x1e\x85\x94R\x94(h\x1e)}\x94h"h\x18sNt\x94bh\x19h\x1ch\x1e\x85\x94R\x94(h\x1e)}\x94h"h\x19sNt\x94bhIhFub\x8c\twildcards\x94h\x06\x8c\tWildcards\x94\x93\x94)\x81\x94}\x94(h\x0e}\x94h\x16]\x94(h\x18h\x19eh\x18h\x1ch\x1e\x85\x94R\x94(h\x1e)}\x94h"h\x18sNt\x94bh\x19h\x1ch\x1e\x85\x94R\x94(h\x1e)}\x94h"h\x19sNt\x94bub\x8c\x07threads\x94K\x01\x8c\tresources\x94h\x06\x8c\tResources\x94\x93\x94)\x81\x94(K\x01K\x01\x8c\x04/tmp\x94e}\x94(h\x0e}\x94(\x8c\x06_cores\x94K\x00N\x86\x94\x8c\x06_nodes\x94K\x01N\x86\x94\x8c\x06tmpdir\x94K\x02N\x86\x94uh\x16]\x94(h\x18h\x19eh\x18h\x1ch\x1e\x85\x94R\x94(h\x1e)}\x94h"h\x18sNt\x94bh\x19h\x1ch\x1e\x85\x94R\x94(h\x1e)}\x94h"h\x19sNt\x94bhkK\x01hmK\x01hohhub\x8c\x03log\x94h\x06\x8c\x03Log\x94\x93\x94)\x81\x94\x8c\x17logs/cnv_facets_bam.log\x94a}\x94(h\x0e}\x94h\x16]\x94(h\x18h\x19eh\x18h\x1ch\x1e\x85\x94R\x94(h\x1e)}\x94h"h\x18sNt\x94bh\x19h\x1ch\x1e\x85\x94R\x94(h\x1e)}\x94h"h\x19sNt\x94bub\x8c\x06config\x94}\x94\x8c\x04rule\x94\x8c\x13test_cnv_facets_bam\x94\x8c\x0fbench_iteration\x94N\x8c\tscriptdir\x94\x8c&/tmp/tmp06v6yc3k/master/bio/cnv_facets\x94ub.'); from snakemake.logging import logger; logger.printshellcmds = True; __real_file__ = __file__; __file__ = '/tmp/tmp06v6yc3k/master/bio/cnv_facets/wrapper.py';
2023-08-30T12:08:03.9187511Z   File "/tmp/tmp06v6yc3k/test/.snakemake/conda/534ad3f46af7d6983c23c4255162115d_/lib/jvm/graalvm-189e927686-java17-22.3.0/languages/python/lib-python/3/pickle.py", line 1757, in _loads
2023-08-30T12:08:03.9188094Z     return _Unpickler(file, fix_imports=fix_imports, buffers=buffers,
2023-08-30T12:08:03.9188849Z   File "/tmp/tmp06v6yc3k/test/.snakemake/conda/534ad3f46af7d6983c23c4255162115d_/lib/jvm/graalvm-189e927686-java17-22.3.0/languages/python/lib-python/3/pickle.py", line 1210, in load
2023-08-30T12:08:03.9189479Z     dispatch[key[0]](self)
2023-08-30T12:08:03.9190428Z   File "/tmp/tmp06v6yc3k/test/.snakemake/conda/534ad3f46af7d6983c23c4255162115d_/lib/jvm/graalvm-189e927686-java17-22.3.0/languages/python/lib-python/3/pickle.py", line 1535, in load_stack_global
2023-08-30T12:08:03.9190973Z     self.append(self.find_class(module, name))
2023-08-30T12:08:03.9191730Z   File "/tmp/tmp06v6yc3k/test/.snakemake/conda/534ad3f46af7d6983c23c4255162115d_/lib/jvm/graalvm-189e927686-java17-22.3.0/languages/python/lib-python/3/pickle.py", line 1577, in find_class
2023-08-30T12:08:03.9192199Z     __import__(module, level=0)
2023-08-30T12:08:03.9192781Z   File "/usr/share/miniconda3/envs/snakemake/lib/python3.11/site-packages/snakemake/__init__.py", line 26, in <module>
2023-08-30T12:08:03.9193406Z     from snakemake.executors.common import url_can_parse
2023-08-30T12:08:03.9194026Z   File "/usr/share/miniconda3/envs/snakemake/lib/python3.11/site-packages/snakemake/executors/__init__.py", line 37, in <module>
2023-08-30T12:08:03.9194622Z     from snakemake.shell import shell
2023-08-30T12:08:03.9195189Z   File "/usr/share/miniconda3/envs/snakemake/lib/python3.11/site-packages/snakemake/shell.py", line 19, in <module>
2023-08-30T12:08:03.9195626Z     from snakemake.deployment import singularity
2023-08-30T12:08:03.9196217Z   File "/usr/share/miniconda3/envs/snakemake/lib/python3.11/site-packages/snakemake/deployment/singularity.py", line 24, in <module>
2023-08-30T12:08:03.9196620Z     class Image:
2023-08-30T12:08:03.9197167Z   File "/usr/share/miniconda3/envs/snakemake/lib/python3.11/site-packages/snakemake/deployment/singularity.py", line 40, in Image
2023-08-30T12:08:03.9197567Z     def hash(self):
2023-08-30T12:08:03.9198081Z   File "/usr/share/miniconda3/envs/snakemake/lib/python3.11/site-packages/snakemake/common/__init__.py", line 209, in __init__
2023-08-30T12:08:03.9198510Z     super().__init__(method, doc=method.__doc__)
2023-08-30T12:08:03.9198960Z AttributeError: 'lazy_property' object has no attribute '__doc__'
2023-08-30T12:08:03.9199277Z [Wed Aug 30 12:07:36 2023]
2023-08-30T12:08:03.9199555Z Error in rule test_cnv_facets_bam:
2023-08-30T12:08:03.9199820Z     jobid: 0
2023-08-30T12:08:03.9200112Z     input: T.sample.bam, N.sample.bam, common.sample.vcf.gz
2023-08-30T12:08:03.9200538Z     output: CNV_bam.vcf.gz, genome_bam.cnv.png, cnv_bam.hist.pdf, qc_bam.spider.pdf
2023-08-30T12:08:03.9200970Z     log: logs/cnv_facets_bam.log (check log file(s) for error details)
2023-08-30T12:08:03.9201484Z     conda-env: /tmp/tmp06v6yc3k/test/.snakemake/conda/534ad3f46af7d6983c23c4255162115d_
2023-08-30T12:08:03.9201864Z Logfile logs/cnv_facets_bam.log not found.```

[fgvieira]

I had the same issue on #1129, and it was because of an old python version.

[tdayris]

Thank you ! Case solved !

I'm using strict channel priority for my local tests. Do you know why I got different version of python ? I thought strict channel priorities ensured an exact reproducibility of conda/mamba environments ?

[fgvieira]

No idea... but conda environments sometimes act a bit funny.