Release notes 2018 08
The 2018-08 release of PhenoMeNal, also known as “Dalcotidine”, was released start of August 2018. The release includes a large number new tools and workflows, including a Galaxy visualization plugin to help users to visualise ISA-Tab datasets within Galaxy. Existing tools and workflows were further improved based on user experience and feedback. All tools and workflows as part of this release have been extensively tested, and documentation and tutorials have been updated and improved. The portal’s interface has been improved thanks to the feedback received from our usability testing sessions. Moreover, we have also implemented the possibility to better customize deployments (e.g., set the number of computing nodes desired) and support for partner cloud providers – which get pre-set connection parameters.
The main user facing components of the release are:
- PhenoMenal Galaxy workflow environment container version v18.01 with >250 tools. Large number of new tools in comparison to the last release.
- Luigi workflow environment container version v2.6.0_cv0.1.7.
- Jupyter Notebooks container version v387f29b6ca83_cv0.4.12.
- Pachyderm workflow environment v1.6.6.
The main deployment components of the release are:
- PhenoMeNal Gateway Portal container version v3.0.1
- Deployment logic cloud-deploy-kubenow (KubeNow wrapper for the EBI Portal) version 0.3
- KubeNow command line client version 0.3.2 with PhenoMeNal plugin KubeNow-plugin
- Galaxy Helm Chart version 1.6.0
The Dalcotidine release of the PhenoMeNal Galaxy container includes:
-
>250 Galaxy tools. Large number of new tools in comparison to the last release, including
- 5 tools for extracting collections of raw data files from an ISA dataset.
- 30 tools for the Eco-Metabolomics workflow targeting data handling and biostatistics
- Existing tools supporting Fluxomics workflow of 13C tracer data analysis have been updated to extend the applicability of the tools. It allows the evaluation of mass isotopomer distribution from more complex mass spectrometer data.
-
3 new pre-assembled Workflows
- ISA-Create-Validate-Upload workflow: The ISA-Create-Validate-Upload workflow is a demonstration of the Galaxy ISA-tools and ISA-Tab Galaxy datatype, utilising the ISA Create tool study designer that generates prospective ISA-Tab templates, validates the new ISA-Tab and generates a validation report, and uploads to MetaboLights Labs if there are no validation errors. The workflow additionally generates a factors summary and visualizes it as a parallel sets plot. This allows users to pre-register a prospective study based on a study plan.
- Eco-Metabolomics workflow: The Eco-Metabolomics workflow is an implementation of a resource demanding use case in the field of ecology, where recently metabolomics methods have been used in large field experiments to describe interactions between different species of organisms in remarkable detail. The workflow is based on MTBLS520: Seasonal variation of secondary metabolites in nine bryophytes. Ecological aspects of the workflow are described in: Peters K, Gorzolka K, Bruelheide H, Neumann S. Seasonal variation of secondary metabolites in nine different bryophytes, [Ecol Evol. 2018;00:1–13] (https://doi.org/10.1002/ece3.4361).
- mzQuality workflow: A set of tools that can be used to assess the quality of targeted mass spectrometry measurements. More information about the tools can be found at mzQuality.nl
This PhenoMeNal Galaxy release provides increased reliability for high loads on Kubernetes clusters by setting resources (CPU/RAM) limits for each tool.
The ISA-Tab Galaxy composite data type that was introduced in the last release has been further enhanced with better preview presentation of the included raw data files.
More details available on the PhenoMeNal Galaxy container README file and on the CI job built.
Additional documentation has been added to the project Wiki including:
- Tutorials for the new [ISA-Create] (https://github.com/phnmnl/phenomenal-h2020/wiki/How-to-create-ISA-metadata-file) and Eco-Metabolomics workflows
- Detailed information on security approach and roadmap, federation technologies and logging and monitoring of PhenoMeNal instances
- Description of tools supporting Fluxomics workflow of 13C tracing data analysis: Ramid, Midcor, Cdt2mid.
- Description of examples of Fluxomic analysis based on the data extracted from literature in collaboration with Molecular Connection: Fluxomics workflow: case studies
- A number of minor changes to Wiki documents have also been made to improve readability.
- A number of tools/containers (available in the App Library) have also had the documentation updated to improve readability.
- Galaxy Tours: A galaxy tour documenting how to use ISAcreate_metabo tool Metfrag-xcms tour
#Video Tutorials The PhenoMeNal YouTube channel includes recordings of webinars that have taken place as part of the PhenoMeNal webinar series. A video tutorial for the ISAcreate metabo galaxy workflow. (https://youtu.be/0HH1g6Bps4s)
The Dalcotidine release includes a Galaxy visualization plugin that integrates the ISA-Tab Viewer. The [ISA-Tab Viewer plugin] (https://github.com/phnmnl/container-galaxy-k8s-runtime/tree/release/v18.01-dalcotidine/config/plugins/visualizations/isatab-viewer) allows us to view an ISA-Tab dataset in a JavaScript-rendered view, with a summary overview of study sample characteristics, study contacts, protocols, and experimental factors, and a clean view of the sample and assay tables.
The Dalcotidine release of the gateway portal puts significant emphasis on improving usability and maintainability. To this end, we ran professional usability testing sessions, which have provided valuable feedback that has been integrated in the portal. In addition, the code has been extensively refactored to favour future maintainability by a broader community. Also in this direction, we have implemented the ability to easily add “partner” OpenStack cloud providers, which are partly preconfigured and thus easier to use.
Here is a short summary of the changes included in this release. Numerous user experience improvements throughout gateway portal Improved App Library navigation and query functionality Improved usability of sign-in/sign-out procedure Advanced deployment parameters Deploy previous releases of PhenoMeNal Configure instance type, cluster and storage size Improved deployment progress monitoring Support for partner cloud providers
Plugin repo system. KubeNow phenomenal specific settings are now a plugin Kn provision and provisioning block in config-file (together with plugin-repo system allows for allows for simple user specific configuration) Logging and monitoring of cluster and pods are done via Elasticsearch, Fluentd, Kibana (EFK) Stack Updated to Kubernetes 1.9.5 version Added support for EGI-VOMS authentication
We have contributed our work on a Galaxy Helm chart for Kubernetes deployments to the wider Galaxy community (available here). Thus, this release now relies on community-supported Helm charts to which we are actively contributing. The new Helm chart supports the new Galaxy deployment architecture, based as well on the docker-galaxy-stable compose community images, used by PhenoMeNal which introduces several important improvements over previous versions. The main ones are: Multi-process Galaxy deployment that brings improved scalability and easily supports large numbers of concurrent users; Secure FTP service to allow upload and download of large datasets With this setup we have verified that the PhenoMeNal VRE works much better in settings with many simultaneous users, such as hands-on workshops.
 |
Funded by the EC Horizon 2020 programme, grant agreement number 654241 |  |
|---|
