Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the definition of source for WorkflowStepInput #247

Open
fmigneault opened this issue Jun 2, 2023 · 1 comment
Open

Improve the definition of source for WorkflowStepInput #247

fmigneault opened this issue Jun 2, 2023 · 1 comment

Comments

@fmigneault
Copy link
Contributor

@mr-c
I found this case:
https://raw.githubusercontent.com/common-workflow-language/cwl-v1.2/1.2.1_proposed/tests/count-lines4-wf.cwl
While working on opengeospatial/ogcapi-processes#329

Which has:

in:
  file1: [file1, file2]

But the doc:
https://www.commonwl.org/v1.2/Workflow.html#WorkflowStep
Shows:
in: array<WorkflowStepInput> | map<id, source | WorkflowStepInput>

While the
outputSource: string | array<string>
is explicitly defined as potentially a string/list of string in https://www.commonwl.org/v1.2/Workflow.html#WorkflowOutputParameter,
the source does not show explicitly the array<string> variant.

There is this part of the text that identifies it as a possibility, but I find that the missing array in the definition makes its somewhat counter intuitive (and in fact, it was not a use case I covered until I got the error because I didn't notice it as something that was possible).

https://www.commonwl.org/v1.2/Workflow.html#Merging_multiple_inbound_data_links
If the sink parameter is an array, or named in a workflow scatter operation, there may be multiple inbound data links listed in the source field. The values from the input links are merged depending on the method specified in the linkMerge field. If both linkMerge and pickValue are null or not specified, and there is more than one element in the source array, the default method is "merge_nested".

I propose to add the array variant explicitly to the table.

@mr-c
Copy link
Member

mr-c commented Jun 5, 2023

The connection between WorkflowStep.in and WorkflowStepInput is defined at

cwl-v1.2/Workflow.yml

Lines 574 to 579 in 4a715dd

- name: in
type: WorkflowStepInput[]
jsonldPredicate:
_id: "cwl:in"
mapSubject: id
mapPredicate: source

The source field is defined at

cwl-v1.2/Workflow.yml

Lines 283 to 293 in 4a715dd

- name: source
doc: |
Specifies one or more workflow parameters that will provide input to
the underlying step parameter.
jsonldPredicate:
"_id": "cwl:source"
"_type": "@id"
refScope: 2
type:
- string?
- string[]?

(It is interesting that Sink is only used by WorkflowStepInput, @tetron is there any reason to not combine them directly? )

The definition of OutputSource is at

cwl-v1.2/Workflow.yml

Lines 228 to 241 in 4a715dd

- name: outputSource
doc: |
Specifies one or more names of an output from a workflow step (in the form
`step_name/output_name` with a `/` separator`), or a workflow input name,
that supply their value(s) to the output parameter.
the output parameter. It is valid to reference workflow level inputs
here.
jsonldPredicate:
"_id": "cwl:outputSource"
"_type": "@id"
refScope: 1
type:
- string?
- string[]?

And it is not involved with any abbreviated form.

If we replace source with string?, string[]? (the type definition of source) in the rendering of the in field then the reader won't have the conceptual meaning of the value.

Options could include

  1. Hyperlinking fields references to their definitions (having those anchors would be very useful, regardless). So both id and source would be hyperlinks to their entries under WorkflowStepInput.
  2. Hover text for field references showing the types (and maybe the label/doc for the field as well?)
  3. Listing the types as a parenthetical

    array<WorkflowStepInput> |
    map<id (string), source (string?, string[]?) | WorkflowStepInput>

  4. Combining 1+2 or 1+3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants