Repository Moved

Important Notice: This project has moved to a new GitHub repository. For the latest version, contributions, and issue tracking, please visit our new repository at AtlantOS QC.

This repository will be maintained as a historical record to highlight the origins of the project and to acknowledge H2020 AtlantOS, now AtlantOS Program for the initial support for the app's development.

Archiving This Repository

This repository will be archived soon, making it read-only. Active development and discussions will continue in the new repository.

AtlantOS QC (GLODAP QC1 tool)

This software is an interactive tool for making the first Quality Control (1st QC, QC1) on Hydrographic Cruise Data. Built mainly in Python but using Node, JavaScript, HTML5 and related technologies. Can be run on most platforms.

About

The development of this tool was initially funded by H2020-AtlantOS project for Optimising and Enhancing the Integrated Atlantic Ocean Observing Systems, being this funded by the European Union's Horizon 2020 research and innovation programme under grant agreement No 633211.

Version 1.0 was released under as Deliverable D2.6 GO-SHIP Software and Manuals: Software packages and best practice manuals and knowledge transfer for sustained quality control of hydrographic sections in the Atlantic of H2020-AtlantOS Work package 2.1, being a Global Ocean Ship-Based Hydrographic Investigations Programme (GO-SHIP) task for support the quality assurance of on-going surveys and perform Quality Control (QC) procedures

Since version 1.0 to 1.4.0, the application had no specific nor continuous funding for its development, having received partial support from H2020-COMFORT project and being also a contribution to WATER:iOS Interdisciplinary Thematic Platform @WATER_IOS_CSIC

From 1.4.0 onwards is supported by EURO GO-SHIP (Grant #101094690), a EU-HORIZON project for Developing a research infraestructure concept to support european hydrography. Current development code is so hosted in https://github.com/EuroGO-SHIP/AtlantOS_QC, but installers will be provided here

Installation

It's not essential for this software to run, but in order to take advantage of the main functionalities of this software program it is absolutly recommended to have GNU Octave installed, as most parameter calculations relies on Octave/Matlab code. Installers can be found here. Latest version proven to work was 5.2, it can be downloaded from here: https://ftp.gnu.org/gnu/octave/

Using built-in installers

• Windows x64: ocean-data-qc-setup-1.4.0.exe
• macOS: ocean-data-qc-1.4.0.dmg
• GNU/linux deb (Ubuntu, Debian,...): ocean-data-qc_1.4.0_amd64.deb
• GNU/Linux rpm (RedHat, SuSe, ...): ocean-data-qc-1.4.0.x86_64.rpm
• GNU/Linux snap: ocean-data-qc_1.4.0_amd64.snap
• GNU/Linux AppImage: ocean-data-qc-1.4.0.AppImage

Source code:

• latest:
• v1.4.0: https://github.com/ocean-data-qc/ocean-data-qc/archive/v1.4.0.tar.gz
• v1.3.0: https://github.com/ocean-data-qc/ocean-data-qc/archive/v1.3.0.tar.gz
• v1.2.0: https://github.com/ocean-data-qc/ocean-data-qc/archive/v1.2.0.tar.gz
• v1.1.1: https://github.com/ocean-data-qc/ocean-data-qc/archive/v1.1.1.tar.gz

Manual Installation

1. Download and install base dependencies:

   1. Download and install Python >=3.8. Recommended downloader and instructions: https://conda.io/miniconda.html

   2. Download and install yarn. Follow instructions from: https://yarnpkg.com/ (Alternatively you can use npm)

   3. Optional but very recommended for functionality: GNU Octave

      • Download and install GNU Octave. Follow instructions from: https://www.gnu.org/software/octave/#install. Latest version proven to work was 5.2, it can be downloaded from here: https://ftp.gnu.org/gnu/octave/

2. Download this project

3. Install the python ocean_data_qc package and its dependencies in your python setup (if you have installed python through miniconda/anaconda and is not in PATH, you have to use Anaconda Prompt as command shell):

    python setup.py develop
        or for non-root install in Linux:
    python setup.py develop --user
   

4. Install the node dependencies in the ocean_data_qc_js folder

    cd ocean_data_qc_js
    yarn install
   

5. Open the GUI from the ocean_data_qc_js folder

    cd ocean_data_qc_js
    yarn start               # or npm start
    (first time launching delays some time, please wait)
   

Short recipe for development version:

0. Install OCTAVE (5.2, no GUI needed)
   • https://ftp.gnu.org/gnu/octave/
1. Install minconda
   • https://docs.conda.io/en/latest/miniconda.html
2. Install git
   • conda install git
3. Install yarn
   • conda install yarn
4. Download code:
   • git clone https://github.com/EuroGO-SHIP/AtlantOS_QC
5. Go to dir:
   • cd AtlantOS_QC
6. Install tool
   • python setup.py develop
   • yarn --cwd ocean_data_qc_js install
7. Launch tool
   • yarn --cwd ocean_data_qc_js start

Libraries and languages Used

• Electron (formerly known as Atom Shell). Electron is an open source library developed by GitHub for building cross-platform desktop applications with HTML, CSS, and JavaScript. Electron accomplishes this by combining Chromium and Node.js into a single runtime and apps can be packaged for Mac, Windows, and Linux. The application uses Electron to embed in an application the bokeh plots and drive the user interaction with interface, providing also all the menus and interfaces.

• Bokeh (Python Library). Bokeh is an interactive visualization library that targets modern web browsers for presentation. Its goal is to provide elegant, concise construction of versatile graphics, and to extend this capability with high-performance interactivity over very large or streaming datasets. Bokeh is the main library for the application, as deals with graphic presentation and interaction (Bokeh Development Team (2014). Python library for interactive visualization. http://www.bokeh.pydata.org)

• Octave. GNU Octave is a high-level interpreted language, primarily intended for numerical computations. It provides capabilities for the numerical solution of linear and nonlinear problems, and for performing other numerical experiments. It also provides extensive graphics capabilities for data visualization and manipulation. The GNU Octave language is quite similar to Matlab™ so that most programs are easily portable. The application uses GNU Octave to drive main oceanographic calculations, as it's actually the main used language in that field of work, and most typical Matlab™ open source oceanographic libraries were built in Matlab™. Running these calculations directly in GNU Octave allows to easy integrating code from researchers.

• Python. Python version 3 is the main language in the application. Most processing on data and files are performed through Pandas and/or NumPy libraries in python, and sent to Bokeh, which is also built on python and has an own javascript library for displaying and interacting. Python source code is presented in ocean_data_qc folder in the application.

• JavaScript. Electron is built on node, that is built with JavaScript. All code for menus, screens, and interacting elements on the application are made with Javascript. Some of the functionality are provided directly by Electron and some other by Bokeh, but the code to merge all these things were built with JavaScript. JavaScript source code is presented in ocean_data_qc_js folder in the application.

License

This project is licensed under the GPLv3 License - see the LICENSE file for details

Authors

• @CSIC: Antón Velo avelo@iim.csic.es
• @CSIC: Jesús Cacabelos jcacabelos@iim.csic.es
• @GEOMAR: Nico Lange nlange@geomar.de
• @CSIC: Fiz F. Pérez fiz.perez@iim.csic.es
• @NORCE: Are Olsen are.olsen@uib.no
• @GEOMAR: Toste Tanhua ttanhua@geomar.de

References

• CO2SYS.m. CO2SYS version 1.1 (Sept 2011)

  van Heuven, S., D. Pierrot, J.W.B. Rae, E. Lewis, and D.W.R. Wallace (2011) MATLAB Program Developed for CO2 System Calculations. ORNL/CDIAC-105b. Carbon Dioxide Information Analysis Center, Oak Ridge National Laboratory, U.S. Department of Energy, Oak Ridge, Tennessee. https://doi.org/10.3334/CDIAC/otg.CO2SYS_MATLAB_v1.1

• CANYON-B. Adapted to run in this application

  Bittig et al. (2018). An alternative to static climatologies: Robust estimation of open ocean CO2 variables and nutrient concentrations from T, S and O2 data using Bayesian neural networks. Front. Mar. Sci. 5:328. http://dx.doi.org/10.3389/fmars.2018.00328.

• Broullón, D., Pérez, F. F., Velo, A., Hoppema, M., Olsen, A., Takahashi, T., Key, R. M., Tanhua, T., González-Dávila, M., Jeansson, E., Kozyr, A.: A global monthly climatology of total alkalinity: a neural network approach, Earth Syst. Sci. Data, 11, 1109-1127, 2019 https://doi.org/10.5194/essd-11-1109-2019

• Velo, A., Pérez, F.F., Tanhua, T., Gilcoto, M., Ríos, A.F., Key, R.M., 2013. Total alkalinity estimation using MLR and neural network techniques. Journal of Marine Systems 111–112, 0, 11–18. https://doi.org/10.1016/j.jmarsys.2012.09.002

• Fernandes, . (2014, August 25). python-seawater v3.3.2 (Version v3.3.2). Zenodo. http://doi.org/10.5281/zenodo.11395