feat: encode fastq downloader

[Maarten-vd-Sande]

Description

Download fastq files directly from the ENCODE project.

Added python as a dependency because wrapper needs an environment: snakemake/snakemake#1718

QC

• I confirm that:

For all wrappers added by this PR,

• there is a test case which covers any introduced changes,
• input: and output: file paths in the resulting rule can be changed arbitrarily,
• either the wrapper can only use a single core, or the example rule contains a threads: x statement with x being a reasonable default,
• rule names in the test case are in snake_case and somehow tell what the rule is about or match the tools purpose or name (e.g., map_reads for a step that maps reads),
• all environment.yaml specifications follow the respective best practices,
• wherever possible, command line arguments are inferred and set automatically (e.g. based on file extensions in input: or output:),
• all fields of the example rules in the Snakefiles and their entries are explained via comments (input:/output:/params: etc.),
• stderr and/or stdout are logged correctly (log:), depending on the wrapped tool,
• temporary files are either written to a unique hidden folder in the working directory, or (better) stored where the Python function tempfile.gettempdir() points to (see here; this also means that using any Python tempfile default behavior works),
• the meta.yaml contains a link to the documentation of the respective tool or command,
• Snakefiles pass the linting (snakemake --lint),
• Snakefiles are formatted with snakefmt,
• Python wrapper scripts are formatted with black.
• Conda environments use a minimal amount of channels, in recommended ordering. E.g. for bioconda, use (conda-forge, bioconda, nodefaults, as conda-forge should have highest priority and defaults channels are usually not needed because most packages are in conda-forge nowadays).

[Maarten-vd-Sande]

@dlaehnemann could you take a look if this is what you had in mind for the seq2science review?

[dlaehnemann]

Yes, this is exactly what I was thinking of. This is really great, many thanks!

I already have some small suggestions, but will have a more thorough tomorrow.

[dlaehnemann]

I think you could work with named outputs here, to avoid relying on the order of specified outputs. Here, I would suggest:

Suggested change

	"{accession}_R1.fastq.gz",
	"{accession}_R2.fastq.gz"
	r1="{accession}_R1.fastq.gz",
	r2="{accession}_R2.fastq.gz"

Below, for the single-end case, I would then suggest something like se= or r0=. Then, you can check for the presence of r1 and r2, or se/r0 in snakemake.output.

[Maarten-vd-Sande]

I'm not sure what to call the single-end named outputs. Based on Illumina's explanation I'm thinking to just name them r1, and r1 + r2 for paired-end: https://knowledge.illumina.com/software/general/software-general-reference_material-list/000002211

For a single-read run, one Read 1 (R1) FASTQ file is created for each sample per flow cell lane. For a paired-end run, one R1 and one Read 2 (R2) FASTQ file is created for each sample for each lane. FASTQ files are compressed and created with the extension *.fastq.gz.

Not entirely convinced of this, but perhaps better than thinking of our own format?

[dlaehnemann]

You are right, from the general logic of Illumina, single-end reads simply sequence read1. se and r0 are just things I have seen in the wild and never questioned. But it sounds better and more consistent to go for your suggestion:

• r1= only for single-end
• r1= and r2= for paired-end

And you can still clearly introduce logic for handling both cases (and throwing an error if neither is the case). Thanks for being so thorough!

[dlaehnemann]

Just another mini-review of this whole environment.yaml conundrum...