pipelines-cwl

• This is a collection of pipelines built by 4DN-DCIC that were used to process 4DN data on the 4DN portal using Tibanna workflow executor.

Directories

• cwl_awsem_v1 : Tibanna AWSEM CWLs in v1.0 in json format

How to run the cwl

To run docker through CWL, you need a cwl executor - we use cwltool (https://github.com/common-workflow-language/cwltool) to run CWL with a json/yml file describing input data. Some example input data are inside the tests/test_input_json directory and you can see some cwltool (=cwl-runner) commands inside the tests/tests.sh script.

Testing CWL

To test cwls in this repo against the test files in tests, use tests/tests_v1.sh with cwl name (without .cwl).

source tests/tests_v1.sh bwa-mem

Notes on releases

Freeze 0.2.7

• All CWL is now in the folder cwl.

The 4DN pipeline suite freeze 0.2.7 is equivalent to the following.

• Hi-C processing pipeline v43
• Repliseq processing pipeline v16
• FastQC v2

Freeze >= 0.2.6

• All CWL is converted to version 1 (no more draft3) and they are kept in the folder cwl_awsem_v1.

Freeze >=0.0.2

• Directory cwl_awsem is manually updated starting from freeze 0.0.2 (after Sep 1, 2017), since we no longer use SevenBridges.

Freeze 0.0.1

• Freeze 0.0.1 contains exported SBG CWLs and AWSEM CWL files auto-converted from SBG CWLs. The following is a description of how freeze 0.0.1 was generated.

  The content has been generated by the following command.

  source download.sh 
  

  For this to work, you need to have SBG_TOKEN as an environmental variable on your machine.

  To convert sbg cwl to awsem cwl in a batch, do the following

  source convert.sh
  

  This conversion uses script convert_sbgcwl_to_awsemcwl.py individually. The shell script assumes Mac (xargs -I{} instead of xargs -i).