Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] --tools is now case insensitive #1188

Open
wants to merge 23 commits into
base: dev
Choose a base branch
from

Conversation

adamrtalbot
Copy link
Contributor

@adamrtalbot adamrtalbot commented Aug 18, 2023

Use new method checkInParam to check if a tool is in --tools or --skip_tools. Will centralise bug fixing etc and make everything easier, i.e. DRY.

Fixes #1187

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/sarek branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@github-actions
Copy link

This PR is against the master branch ❌

  • Do not close this PR
  • Click Edit and change the base to dev
  • This CI test will remain failed until you push a new commit

Hi @adamrtalbot,

It looks like this pull-request is has been made against the adamrtalbot/sarek master branch.
The master branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to master are only allowed if they come from the adamrtalbot/sarek dev branch.

You do not need to close this PR, you can change the target branch to dev by clicking the "Edit" button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.

Thanks again for your contribution!

@adamrtalbot adamrtalbot changed the base branch from master to dev August 18, 2023 13:23
Copy link
Contributor

@FriederikeHanssen FriederikeHanssen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice ❤️

CHANGELOG.md Outdated Show resolved Hide resolved
@@ -24,6 +24,7 @@ jobs:
id: filter
with:
filters: "tests/config/tags.yml"
token: ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an attempt to reduce the amount of API calls by pulling the repo down and performing a git diff. At least that's what the docs say but I don't think it worked.

@maxulysse
Copy link
Member

You need to update schema as well

Co-authored-by: Friederike Hanssen <Friederike.hanssen@qbic.uni-tuebingen.de>
@github-actions
Copy link

github-actions bot commented Aug 19, 2023

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 81462b2

+| ✅ 140 tests passed       |+
#| ❔   9 tests were ignored |#
!| ❗   4 tests had warnings |!

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 2.9
  • Run at 2023-09-07 11:33:51

Copy link
Contributor

@asp8200 asp8200 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about making the change just one place centrally? Something like

params.tools = params.tools.toLowerCase()

in sarek.nf? Then there should be no danger of me forgetting to add .toLowerCase() when doing a contains() on, say params.tools.split(',') :-)

@maxulysse
Copy link
Member

How about making the change just one place centrally? Something like

params.tools = params.tools.toLowerCase()

in sarek.nf? Then there should be no danger of me forgetting to add .toLowerCase() when doing a contains() on, say params.tools.split(',')`:-)

we cannot change params

@maxulysse
Copy link
Member

How about making the change just one place centrally? Something like
params.tools = params.tools.toLowerCase()
in sarek.nf? Then there should be no danger of me forgetting to add .toLowerCase() when doing a contains() on, say params.tools.split(',')`:-)

we cannot change params

What could be done instead, would be to use a variable for that, which is what we where doing pre-nf-core, but with DSL2 and the way we deal with the modules in the config files, it'll quickly become a mess.

@asp8200
Copy link
Contributor

asp8200 commented Aug 21, 2023

How about making the change just one place centrally? Something like
params.tools = params.tools.toLowerCase()
in sarek.nf? Then there should be no danger of me forgetting to add .toLowerCase() when doing a contains() on, say params.tools.split(',')`:-)

we cannot change params

What could be done instead, would be to use a variable for that, which is what we where doing pre-nf-core, but with DSL2 and the way we deal with the modules in the config files, it'll quickly become a mess.

Hmm ... How about this some place centrally:

tools = params.tools.split(',').toLowerCase()

and then use do a search-replace params.tools.split(',').toLowerCase() all over this PR?

@maxulysse
Copy link
Member

How about making the change just one place centrally? Something like
params.tools = params.tools.toLowerCase()
in sarek.nf? Then there should be no danger of me forgetting to add .toLowerCase() when doing a contains() on, say params.tools.split(',')`:-)

we cannot change params

What could be done instead, would be to use a variable for that, which is what we where doing pre-nf-core, but with DSL2 and the way we deal with the modules in the config files, it'll quickly become a mess.

Hmm ... How about this some place centrally:

tools = params.tools.split(',').toLowerCase()

and then use do a search-replace params.tools.split(',').toLowerCase() all over this PR?

I don't really like it as it obfuscates even more in the config where this tools is coming (also not even sure it'll work properly).

@adamrtalbot adamrtalbot changed the title --tools is now case insensitive [WIP] --tools is now case insensitive Aug 21, 2023
@adamrtalbot
Copy link
Contributor Author

I don't really like it as it obfuscates even more in the config where this tools is coming (also not even sure it'll work properly).

I was considering some method for checking in tools, e.g.:

checkInParam(params.tools, "mutect2")

But I'm not sure this is any better. Let's mark this as WIP while we try stuff out.

@maxulysse
Copy link
Member

I don't really like it as it obfuscates even more in the config where this tools is coming (also not even sure it'll work properly).

I was considering some method for checking in tools, e.g.:

checkInParam(params.tools, "mutect2")

But I'm not sure this is any better. Let's mark this as WIP while we try stuff out.

I like that idea

@adamrtalbot
Copy link
Contributor Author

I've implemented it as checkTools method to be reused in configuration. Let's see if it works 🤞

Copy link
Member

@maxulysse maxulysse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it

@@ -1215,7 +1215,7 @@ workflow SAREK {
// ANNOTATE
if (params.step == 'annotate') vcf_to_annotate = input_sample

if (params.tools.split(',').contains('merge') || params.tools.split(',').contains('snpeff') || params.tools.split(',').contains('vep')) {
if (checkTools(params.tools, 'merge') || checkTools(params.tools, 'snpeff') || checkTools(params.tools, 'vep')) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have checkTools(params.tools, ['merge', 'snpeff', 'vep']) or something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we could - I just did find + replace. Once I can get the tests working I'll tidy it up.

workflows/sarek.nf Outdated Show resolved Hide resolved
workflows/sarek.nf Outdated Show resolved Hide resolved
@@ -1308,6 +1308,13 @@ def flowcellLaneFromFastq(path) {
return fcid
}

// Check the parameters tools or skip_tools, then compare it against the provided tool
// Returns true/false based on whether 'tool' is found in 'parameter'
def checkTools(parameter, tool) {
Copy link
Contributor

@asp8200 asp8200 Aug 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems like the function checkTools could also be used for other parameters which don't have anything to do with tools (like say params.aligner or params.step), so perhaps the function might as well be called something more general like contains(if we - and nextflow - can handle that kind of "name" overloading) or your original proposal checkInParam?

Anyways, it's just a thought, and we can also just go with checkTools for now, and see which way the wind blows.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm good idea.

// Check the parameters tools or skip_tools, then compare it against the provided tool
// Returns true/false based on whether 'tool' is found in 'parameter'
def checkTools(parameter, tool) {
parameter.split(',').any{ it.toLowerCase().contains(tool) }
Copy link
Contributor

@asp8200 asp8200 Aug 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about using equals() instead of contains()? In the following example, the variable X is true and wrongly indicate that manta is among the requested tools.

tools = 'strelka,manta2'
X = tools.split(',').any{ it.toLowerCase().contains('manta') }
Y = tools.split(',').any{ it.toLowerCase().equals('manta') }

The variable Y, on the other hand, is false and correctly indicate that manta is not among the requested tools. (I know we don't have a "manta2", but you get the point.)

Also, if you go with equals() then I also suggest callingtrim() before calling equals(). The reason being that the user might add a space in the list of tools - like this:

tools = 'strelka, manta'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

@adamrtalbot
Copy link
Contributor Author

adamrtalbot commented Aug 29, 2023

Current status, this works everywhere using the basic configuration, but fails everywhere that the config is a few levels deep because it fails to inherit the method properly. Some investigating into the scoping is required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improvement: make --tools case insensitive
4 participants