Expressions in Janis #10
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Conflicts: # janis_core/types/selectors.py # janis_core/utils/__init__.py # janis_core/workflow/workflow.py
# Conflicts: # janis_core/types/selectors.py
# Conflicts: # janis_core/translations/cwl.py
They should've been breaking before...
# Conflicts: # janis_core/tests/test_translation_wdl.py # janis_core/translations/cwl.py # janis_core/translations/wdl.py # setup.py
|
This is looking pretty good. I've got some basic docs up:Janis: Expressions. Because we've introduced the
|
Closed
|
More thoughts, I've yet to categorise these yet:
|
# Conflicts: # janis_core/tests/test_translation_cwl.py
# Conflicts: # janis_core/graph/steptaginput.py # janis_core/tests/test_translation_cwl.py # janis_core/tests/test_translation_wdl.py # janis_core/translations/cwl.py # janis_core/translations/wdl.py # janis_core/workflow/workflow.py # setup.py
# Conflicts: # janis_core/tool/commandtool.py # janis_core/translations/cwl.py
(implementation for CWL still in testing)
|
Cromwell PR has been merged 🎉 !! This PR has been folded into #38, so see that for more information. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR addresses and closes:
Most workflow specifications allow you to transform some value:
We want to support at least some of this functionality. CWL affords us much more flexibility as we could generate javascript functions, WDL is a little stricter in which we would really only implement the standard library for now.
One generic way might be to create a "JavaScript tool", and then we could share generated javascript expressions between CWL and WDL. This would allow us to very flexibly go outside the standard library functions, but adds complexity and potential costs to perform this computation (depending on how it's executed).
Requirements
Considerations
As part of this type system, we might reconsider adding a check between the selector and the ToolOutput type (to make sure they're valid).
Guide
Janis now supports an
Operatorto transform some value from aSelector, or the result from anotherOperator. You could see an operator as a function, but that might include adding, subtracting, and so on.Where possible, we've tried to override the inbuilt python functions. You can see how we could prefix the value from the input node
w.input_valuewith"prefix".Note, this doesn't right now change the value of the input node, it builds up a set of operations (
AddOperator("prefix", InputSelector("input_value"))), which then get turned into either javascript or WDL functions.Applying the representation function of a single operator (
operator.__repr__()) will give you pseudocode of the operation you're going to implement.WIP
This PR is work in progress, there is moderate support for WDL currently, and the type system is lacking.