Package installation

[grst]

Maybe not relevant yet, but I wanted to put some thoughts on this together.

Omnideconv has quite complex dependencies (R packages, python, and docker(/podman/singularity), so we need to make sure it is still easy to install. In principle, there are three installation routes:

1. Plain R package

remotes::install_github("something/omnideconv")

(CRAN/Bioconductor won't be an option, as too many dependencies are not available there)

This will only install R packages, so we need to address python and docker.

• I don't think we can automatically install docker, so this needs to be documented. It would also be nice to support other container engines, as docker requires sudo and is therefore not suitable for everyone (see also Docker alternatives for cibersortx #39). If docker is not available, the other methods should still work fine.
• For python packages, basilisk is an interesting option on top of reticulate. It sets up an independent miniconda installation for the python dependencies.

2. Bioconda

conda install -c bioconda -c conda-forge r-omnideconv

This will install both R and python packages. There is no need to use basilisk or reticulate to install packages.
The challenge here is that some of our dependencies are not available from bioconda (those in the Remotes section). We will
need to contribute those packages to bioconda before adding our package. I had to do the same for immunedeconv.

3. Container

# this should start an R shell with omnideconv available
podman run -it quay.io/biocontainers/r-omnideconv R

As soon as omnideconv is on bioconda, we get a container for free from Biocontainers :).
The question is how to run cibersort in that case. Maybe docker in docker (dind) could work.
It may be that we need to manually extend the container from Biocontainers for that.

[grst]

List of all dependencies (from DESCRIIPTION). Add checkmark if they are on bioconda:

Imports:

• anndata (r-anndata is hosted here, should be added to conda-forge instead of bioconda)
• assertthat,
• Biobase,
• dplyr,
• ggplot2,
• methods ( part of bas R or r-r.methodss3 (?)),
• RColorBrewer,
• readr,
• remotes,
• S4Vectors,
• Seurat,
• SingleCellExperiment,
• stringr,
• SummarizedExperiment,
• tidyr,
• utils (part of base R (?) )

Suggests:

• BisqueRNA (PelzKo/bisque) --> on bioconda, but not the patched version!
• bseqsc (shenorrlab/bseqsc,)
• CDSeq (PelzKo/CDSeq_R_Package)
• DWLS (PelzKo/dwls)
• devtools,
• e1071,
• knitr,
• limSolve,
• magrittr,
• MAST,
• MOMF (grst/MOMF,)
• MuSiC (xuranw/MuSiC)
• pkgdown,
• quadprog,
• reshape,
• reticulate,
• rmarkdown,
• ROCR,
• scBio (amitfrish/scBio)
• SCDC (grst/SCDC,)
• testthat (>= 2.0.0),
• varhandle

Python dependencies

• autogenes
• scaden
• (anndata), the anndata r-package is a reticulate wrapper

[grst]

lol, do I get that right that bseqsc and scbio are the only unpatched deconvolution dependencies?

@PelzKo, could you please add all python dependencies to that list?

[PelzKo]

lol, do I get that right that bseqsc and scbio are the only unpatched deconvolution dependencies?

MuSiC is also unpatched, IF they accept my pullrequest (to allow a single bulk data sample to be used). But yes, otherwise thats true... Which makes our package quite helpful I think.
Also these are just the R-packages, CibersortX, AutoGeneS and Scaden are working as intended

@PelzKo, could you please add all python dependencies to that list?

Done

[grst]

Thanks!

List of patches:

• CDSeq: CDSeq is not patched, it is just updated to use a different version of immunogenomics/harmony. We patched harmony to correctly import an overloaded operator.
• MOMF: patched to add missing dependencies
• MuSiC: patched to fix single sample bug
• Bisque: We edited the deconvolute method to accept a foreign signature matrix
• SCDC: patched to fix remote/bioconductor dependencies

In summary, I think Bioconda could be feasible. The patches for missing dependencies are irrelevant for bioconda,
since bioconda does not use R's dependency resolution anyway. Music's single sample bug is something that we can live with for now. That would mean the only issue would be harmony, but I'm confident it will be eventually resolved.

[grst]

@PelzKo, could you please link all the upstream issues/PRs in this thread just to keep an overview...

[PelzKo]

Merged ones will be striked through

r-anndata -> bioconda pull request is created
BisqueRNA (PelzKo/bisque) --> on bioconda, but not the patched version! Pull Request is not being answered as per #86, we don't need that anymore
DWLS (PelzKo/dwls) --> Pull Request is not being answered, bioconda pull request is created
bseqsc (shenorrlab/bseqsc) -> bioconda pull request is created; csSAM needs to be added first
CDSeq (PelzKo/CDSeq_R_Package) -> no Release, Issue is opened; harmony still needs fixing, but pull request is not being answered; bioconda pull request without release is opened
MOMF (grst/MOMF) -> no Release, Issue is opened; bioconda pull request without release is opened
MuSiC (xuranw/MuSiC) -> no Release, Issue is opened; patched to fix single sample bug, no response yet; bioconda pull request without release is opened
scBio (amitfrish/scBio) -> bioconda pull request for bioconda is done, just not merged yet
SCDC (grst/SCDC) -> no Release, Issue is opened; Pull reuquest opened; also requires L1pack which requires heavy; bioconda pull request without release is opened

Python dependencies

autogenes -> bioconda pull request is created

[grst]

I had another go at the bioconda recipes... but maybe we should just release a docker container and be done with it.

[grst]

I think I figured out MOMF.

This leaves only two problematic PRs: csSAM and SCDC.
I don't have the foggiest how to solve the environment not resolving on the bioconda CI, but I'd suggest to try moving those two packages to conda-forge.

Conda-forge has a different build system in general, and uses mamba instead of conda since a few weeks, so we have better chances.

@PelzKo, do you still have capacity to try this in the next weeks?

[PelzKo]

I think I figured out MOMF.

This leaves only two problematic PRs: csSAM and SCDC. I don't have the foggiest how to solve the environment not resolving on the bioconda CI, but I'd suggest to try moving those two packages to conda-forge.

Conda-forge has a different build system in general, and uses mamba instead of conda since a few weeks, so we have better chances.

@PelzKo, do you still have capacity to try this in the next weeks?

Yeah I can do that. Thanks for your work with MOMF!
I will try to do csSAM and SCDC next week

[grst]

I just checked again, and conda-forge won't work because both packages have dependencies that are only on bioconda (bioconductor-...) 😅

…

On Tue, 7 Dec 2021 at 04:35, Konstantin Pelz ***@***.***> wrote: I think I figured out MOMF. This leaves only two problematic PRs: csSAM and SCDC. I don't have the foggiest how to solve the environment not resolving on the bioconda CI, but I'd suggest to try moving those two packages to conda-forge. Conda-forge has a different build system in general, and uses mamba instead of conda since a few weeks, so we have better chances. @PelzKo <https://github.com/PelzKo>, do you still have capacity to try this in the next weeks? Yeah I can do that. Thanks for your work with MOMF! I will try to do csSAM and SCDC next week — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#57 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVZRVZT23YH4SH2LJ7SJSLUPV6GXANCNFSM46FVY4DA> .

[PelzKo]

Love it... What would you suggest? Should I try tagging the core team again or is there another developer who could look it over? Unfortunately it is the first time I'm working with conda, bioconda and conda-forge, so I am not sure of how much help I can be

[grst]

Ok, I got them all in!
I don't know what the actual problem was, but by requiring a relative recent version for all packages the constraints for the conda solver were reduced and there were no conflicts or timeouts anymore.

Next step would be building the actual omnideconv package. We can try locally with conda build, but for putting it on bioconda we would first need to make the code public.

[grst]

Now that the repo is public, we need to create a bioconda recipe for omnideconv itself!