WDL Scala Tools

Scala programming language library for parsing WDL, and command-line tools for type-checking, code formatting, and more.

Abstract Syntax Tree (AST)

The wdlTools.syntax package implements an abstract syntax tree (AST) for WDL. It uses the OpenWDL Antlr4 grammar originally created by Patrick Magee.

Currently, WDL draft-2, 1.0, 1.1, and development/2.0 are fully supported (with some limitations).

import wdlTools.syntax.AbstractSyntax._
import wdlTools.syntax.Parsers
import dx.util.{FileNode, FileSourceResolver}
val parsers = Parsers(followImports = true)
val wdl: FileNode = FileSourceResolver.get.resolve("file:///path/to/my/wdl")
val doc: Document = parsers.parseDocument(wdl)
// print the source locations of all tasks in the document
doc.elements.foreach {
  case task: Task => println(s"Task ${task.name} is at ${task.loc}")
}

Command line tools

The wdlTools JAR also provides various command line tools to accelerate and simplify the development of WDL workflows. You can get help by running:

$ java -jar wdlTools.jar [command] --help

The following commands are currently available. They should be considered "beta" quality - please report any bugs/enhancements using the issue tracker.

• check: type-check a WDL file
• docgen: generate documentation for WDL tasks/workflows
• exec: execute a WDL task (workflow execution is not yet supported)
• fix: attempts to fix non-standard WDL files with specification incompatibilities
• format: reformat a WDL file
  • Note: draft-2 is not yet supported.
• lint: detect "lint" (i.e. incorrect style or potentially problematic code) in a WDL file
  • Note: this command is not yet full-featured. You may want to also use miniwdl lint.
• new: generate a new WDL project
• printTree: print the Abstract Syntax Tree for a WDL document
• readmes: generate README files for the tasks/workflows in a WDL file - these files are named so that they will be recognized when building DNAnexus apps/workflows using dxCompiler
• upgrade: upgrade a WDL file to a newer version; currently only draft-2 -> [1.0, 1.1] is supported

Limitations

• Forward references are not yet supported, i.e. a variable must be declared before it is referenced
• Incomplete linter rules
• Runtime and hint attributes cannot be overriden at runtime in the task runner

Building

The java code for the parser is generated by the ANTRL4 tool.

1. Download a jar file with the instructions
2. cd GIT_REPO
3. Assuming you downloaded the jar file to $HOME/antlr-4.9-complete.jar generated java code from the grammar by running make from the toplevel.
4. Build the scala code:

   sbt compile
   

5. Run the tests:

   sbt test
   

See the documentation for full details.

Support

wdlTools is not an official product of DNAnexus. Please do not contact DNAnexus (or any employees thereof) for support. To report a bug or feature request, please open an issue in the issue tracker.