Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding cellrangermulti subworkflow #276

Merged
merged 160 commits into from May 22, 2024
Merged

Adding cellrangermulti subworkflow #276

merged 160 commits into from May 22, 2024

Conversation

fmalmeida
Copy link
Contributor

@fmalmeida fmalmeida commented Nov 23, 2023 •
edited by grst

Close #247
Close #313

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/scrnaseq branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

Context

Hi guys,

Although not finished yet because it would still required updating the parameters schema, defaults and documentation, I am already opening the PR so we can all take a look at it and discuss any modifications required before merging and also, give it a round of tests and define how we want some parameters to be.

I used the templates provided by @klkeys

Usage context
image

samplesheet
To use it, samplesheet requires an additional parameter so that we can properly mix the different feature types given per sample.

sample,fastq_1,fastq_2,feature_type,protocol,expected_cells
PBMC_10K,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/5gex/5gex/subsampled_sc5p_v2_hs_PBMC_10k_5gex_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/5gex/5gex/subsampled_sc5p_v2_hs_PBMC_10k_5gex_S1_L001_R2_001.fastq.gz,gex,SC5P-PE,1000
PBMC_10K,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/bcell/subsampled_sc5p_v2_hs_PBMC_10k_b_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/bcell/subsampled_sc5p_v2_hs_PBMC_10k_b_S1_L001_R2_001.fastq.gz,vdj,SC5P-PE,1000
PBMC_10K,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/5gex/5fb/subsampled_sc5p_v2_hs_PBMC_10k_5fb_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/5gex/5fb/subsampled_sc5p_v2_hs_PBMC_10k_5fb_S1_L001_R2_001.fastq.gz,ab,SC5P-PE,1000
PBMC_10K_CMO,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc_cmo/fastqs/gex_1/subsampled_SC3_v3_NextGem_DI_CellPlex_Human_PBMC_10K_1_gex_S2_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc_cmo/fastqs/gex_1/subsampled_SC3_v3_NextGem_DI_CellPlex_Human_PBMC_10K_1_gex_S2_L001_R2_001.fastq.gz,gex,SC3Pv3,1000
PBMC_10K_CMO,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc_cmo/fastqs/cmo/subsampled_SC3_v3_NextGem_DI_CellPlex_Human_PBMC_10K_1_multiplexing_capture_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc_cmo/fastqs/cmo/subsampled_SC3_v3_NextGem_DI_CellPlex_Human_PBMC_10K_1_multiplexing_capture_S1_L001_R2_001.fastq.gz,cmo,SC3Pv3,1000
PBMC_10K_CMV,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/gex_1/subsampled_5k_human_antiCMV_T_TBNK_connect_GEX_1_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/gex_1/subsampled_5k_human_antiCMV_T_TBNK_connect_GEX_1_S1_L001_R2_001.fastq.gz,gex,SC5P-R2,1000
PBMC_10K_CMV,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/ab/subsampled_5k_human_antiCMV_T_TBNK_connect_AB_S2_L004_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/ab/subsampled_5k_human_antiCMV_T_TBNK_connect_AB_S2_L004_R2_001.fastq.gz,ab,SC5P-R2,1000
PBMC_10K_CMV,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/vdj/subsampled_5k_human_antiCMV_T_TBNK_connect_VDJ_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/vdj/subsampled_5k_human_antiCMV_T_TBNK_connect_VDJ_S1_L001_R2_001.fastq.gz,vdj,SC5P-R2,1000

Supporting files

Right now, all the supporting files have been added as parameters, for example, cmo_barcode_csv, beam_antigen_csv, etc. .... which means they will work in a dataset manner, being the same for everything given in the samplesheet, instead of samplesheet base if they were added as columns in the samplesheet.

My main question here is, what should it be the desired approach?

Other stuff
Of course there might still have things to clear or finish that I might have overlooked since there is quite a lot on it, so, I request your help on spotting it.

testing case required the full genomes from ensembl, otherwise, the analysis using VDJ was failling.

fmalmeida and others added 29 commits November 7, 2023 11:31
@fmalmeida
Add cellranger multi testing assets
94397e5
@fmalmeida
allow cellrangermulti option
e6a54bb
@fmalmeida
include cellrangermulti testing conf/profile
ae20b9e
@fmalmeida
allow cellrangermulti option
5863d2c
@fmalmeida
fix example samplesheet
631a980
@fmalmeida
fixed samplesheet for cellranger multi
70fff60
@fmalmeida
don't get cellrangemulti metadata if not needed
f97e430
@fmalmeida
fix check_samplesheet script to be more generic
ff66e97
@fmalmeida
update input_check for cellranger multi
0af2761
@fmalmeida
avoid renaming sample ids in input check
8e7d436
@fmalmeida
generate a parsed input channel for cellrangermulti sub-workflow
97281aa
@fmalmeida
defined cellrangemulti sub-workflow and parsed input channel for exec…
8b19f48 
…ution
@fmalmeida
included gex (normal) reference building and updated cellranger modules
6f494fe
@fmalmeida
include mkvdjref
215226b
@fmalmeida
refactored sample mapping
7b86f80
@fmalmeida
finally cellranger multi running, with errors, but now can be debugged
e6dc3b2
@fmalmeida
not finding samples in data directory
7c66115
@fmalmeida
saving quick changes for shifting development workspace
abb0e3c
include option for unzipping reference files
5e27a4c
@fmalmeida
First successfull run of cellranger multi with renaming module
e00a78d
@fmalmeida
add whiteline
ead6462
Testing github traffic
ab4425e
Remove file used for testing
2cfb148
input dataset parsing refactored and fixed
4c275b4
include cellrangermulti outputs in mqc channel
a8d2702
include option for cellrangermulti in mtx conversion modules
0d8be69
add files filter for cellranger multi outputs
4d75c83
include cellranger multi outputs to mtx conversion subworkflow
e4e37b5
update changelog
132e247
grst
grst reviewed May 15, 2024
View reviewed changes
docs/usage.md Show resolved Hide resolved
@grst
Copy link
Member

grst commented May 15, 2024

Ok. I think I'm done with the documentation.

fmalmeida
fmalmeida commented May 16, 2024
View reviewed changes
docs/usage.md Outdated Show resolved Hide resolved
@grst
Copy link
Member

grst commented May 17, 2024

I don't think lint will pass before the template update is merged. However, since this is such massive PR, I'd do that after this one gets merged to dev to avoid further disruptions.

grst
grst approved these changes May 17, 2024
View reviewed changes
Copy link
Member

@grst grst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we there then? I don't think there's anything pending for now.

@fmalmeida
Copy link
Contributor Author

Are we there then? I don't think there's anything pending for now.

Hi @grst,

I think so. The last things were the documentation and the things you had brought.
There are a few TODOs but can be taken later. As the BEAM data.

This was the comment summarising it: #276 (comment)

We can have a last look on that, and open follow-ups, otherwise, it is.

@maxulysse maxulysse merged commit 7286aa6 into dev May 22, 2024
13 checks passed
@maxulysse maxulysse deleted the 247-support-for-10x-ffpe-scrna branch May 22, 2024 17:35
@fmalmeida fmalmeida mentioned this pull request May 23, 2024
Open
@fmalmeida
Copy link
Contributor Author

I have added a follow-up ticket for the open points: #332

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@klkeys klkeys klkeys left review comments

@apeltzer apeltzer apeltzer approved these changes

@grst grst grst approved these changes

Assignees

@fmalmeida fmalmeida

Labels
Ready for review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

allow use of .gz when specifying fast and gtf Support for 10x FFPE scRNA
6 participants
@fmalmeida @grst @apeltzer @NNazeu @klkeys @maxulysse