Skip to content

87

Latest
Latest
Compare
Choose a tag to compare
@aednichols aednichols released this 07 May 17:47
· 25 commits to develop since this release
87
f8c9ff2

87 Release Notes

GCP Batch

  • Added Nvidia driver install (default 418) (#7235)
  • Fixed Docker mounting volumes with extra colon (#7240)
  • Fixed issue with multiple zones defined in config (#7240)
  • Fixed Batch label regex (#7355)

Progress toward WDL 1.1 Support

WDL 1.1 support is in progress. Users that would like to try out the current partial support can do so by using
WDL version development-1.1. As of Cromwell 87, development-1.1 includes:

  • Engine functions:
    • Added suffix (#7363)
    • Added unzip (#7363)
    • Added quote and squote (#7375)
    • Updated sub to expect POSIX-flavor regex (#7374)
  • Struct literals can be included in WDLs (#7391) (#7402)
  • Added returnCodes runtime attribute (#7389)

upgrade command removed from Womtool

Womtool previously supported a womtool upgrade command for upgrading draft-2 WDLs to 1.0. With WDL 1.1 soon to
become the latest supported version, this functionality is retiring. (#7382)

Replacement of gsutil with gcloud storage

In this release (#7359), all localization functionality on the GCP backend migrates to use the more modern and performant gcloud storage. With sufficiently powerful worker VMs, Cromwell can now localize at up to 1200 MB/s [0][1][2].

In a future release, delocalization will also migrate to gcloud storage. As part of that upcoming change, we are considering turning on parallel composite uploads by default to maximize performance. Delocalized composite objects will no longer have an md5 checksum in their metadata; refer to the matrix below [3]. If you have compatibility concerns for your workflow, please submit an issue.

Delocalization Strategy Performance crc32c md5
Classic Baseline/slow ✅ ✅
Parallel Composite Fast ✅ ❌

[0] Tested with Intel Ice Lake CPU platform, 16 vCPU, 32 GB RAM, 2500 GB SSD

[1] Throughput scales with vCPU count with a plateau at 16 vCPUs.

[2] Throughput scales with disk size and type with at a plateau at 2.5 TB SSD. Worked example: 1200 MB/s ÷ 0.48 MB/s per GB = 2500 GB.

[3] Cromwell itself uses crc32c hashes for call caching and is not affected

Other Improvements

  • In certain cases DRS downloads have been found to hang forever. Cromwell will now time these out. (#7416)
  • Increased default Akka client.parsing.max-response-reason-length to 1024 (#7406)
  • Workflow Completion Callback bodies now include fully-qualified output names (#7234)
  • Improved workflow abort error handling (#7245)
  • Improved logging for troubleshooting (#7246) (#7253) (#7388)
  • Support for Intel Ice Lake chips in Life Sciences backend (#7252)
  • Fix workflows getting stuck in Aborting when WDL has a type error (#7385)
  • Updates to dependencies to fix security vulnerabilities.
Assets 4