Feature Request: Download reference genome and known variation

[pettyalex]

Hello,

I've been looking around at published complete variant calling pipelines, and grenepipe seems both high quality and very flexible.

It would be nice if there were the option to have grenepipe download reference genomes and known variation based on config, if one wanted complete reproducibility about which specific build was used for an analysis. The older, unmaintained https://github.com/snakemake-workflows/dna-seq-gatk-variant-calling did this and we could basically lift their implementation if you're up for it.

Theirs looks like this:
https://github.com/snakemake-workflows/dna-seq-gatk-variant-calling/blob/main/config/config.yaml
https://github.com/snakemake-workflows/dna-seq-gatk-variant-calling/blob/main/workflow/rules/ref.smk

I'd be glad to be the one to implement this if you are interested in it.

[lczech]

Hi @pettyalex,

that sounds like a useful addition! I think it makes sense to offer that as an alternative to specifying those files directly, yes.

Looking at the links you provided, it seems this would be a download from ensembl, right? Or any other databases as well? I've already implemented parts of ensembl downloads for the VEP annotation, see here for the download rules and here for the config. I think to keep matters separate, that part should be kept as-is, but could serve as inspiration for the code as well. As far as I remember, I had quite some trouble with the URL stability at ensembl, and needed all the trickery in these files to get it to work for species such as A. thaliana, which were somehow not in the main database, but needed some special URLs.

If you want to implement this, that would be super cool! I'd suggest to extend the config file maybe like that:

data:
    reference-genome: "/path/to/data/genome.fa"

    # Automatic download from ensembl, should the above file not yet exist:
    reference-genome-download:
        # ensembl species name
        species: homo_sapiens
        # ensembl release
        release: 98
        # genome build
        build: GRCh38
        # TODO need special URL for non-metazoans
        cache-url: ""

That way, the rest of the pipeline could be kept with minimal changes, and the automatic download would store the file in the location as indicated by reference-genome. That works nicely with Snakemake as well, which would then only execute the download if that file is not yet present. An additional rule file with rules similar to the ones you linked would then do that job.

In that config example, I've also added the cache-url part that I had to do for VEP. See the config part that I linked about for details. I think that should be included here as well then, for completeness.

What do you think, is that what you had in mind? Happy to discuss a different setup as well, if you think that would make more sense.

Cheers and so long
Lucas