Copyright (c) 2020 lnferris
ocean_data_tools simplifies the process of extracting, formatting, and visualizing freely-available oceanographic data. While a wealth of oceanographic data is accessible online, some end-users may be dissuaded from utilizing this data due to the overhead associated with obtaining and formatting it into usable data structures. ocean_data_tools solves this problem by allowing the user to transform common oceanographic data sources into uniform structs, call generalized functions on these structs, easily perform custom calculations, and make graphics.
Find a bug, have a question, or want to chat about contributing? Open an issue or email lnferris@alum.mit.edu.
- Download bathymetry.
- Download nctoolbox. You will need to run the command
setup_nctoolboxat the beginning of each MATLAB session. - Add ocean_data_tools and nctoolbox to the path. Specifically, the following folders must be added to the path:
- ocean_data_tools/ocean_data_tools
- ocean_data_tools/ocean_data_tools/utilities
- nctoolbox/
- Run each demonstration in demos/demos.m, which contains example usages for all functions. All required test data is included in data/.
Functions are named using a two-part system. The prefix (argo_, bathymetry_, general_, etc.) indicates the appropriate data source, while the suffix (\_build, \_profiles, \_section, etc.) indicates the action performed. Functions with the \_build suffix load raw data into uniform structs (e.g. argo, cruise, hycom, mercator, woa, wod). Uniform structs created by \_build functions are compatable with any general_ function.
Data sources currently supported:
| Data Source | DOI, Product Code, or Link |
|---|---|
| Argo floats | doi:10.17882/42182 |
| Smith & Sandwell bathymetry | doi:10.1126/science.277.5334.1956 |
| IOOS Glider DAC | https://gliders.ioos.us/ |
| MOCHA Climatology | doi:10.7282/T3XW4N4M |
| HYbrid Coordinate Ocean Model | https://hycom.org |
| CMEMS Global Ocean 1/12° Physics Analysis and Forecast | GLOBAL_ANALYSIS_FORECAST_ PHY_001_024 |
| CMEMS Global Ocean Waves Multi Year | GLOBAL_REANALYSIS_WAV_001_032 |
| GO-SHIP hydrographic cruises | https://www.go-ship.org/ |
| World Ocean Atlas 2018 | https://www.ncei.noaa.gov/products/world-ocean-atlas |
| World Ocean Database | https://www.ncei.noaa.gov/products/world-ocean-database |
Main functions are located in ocean_data_tools/. Demonstrations are located in demos/. Test datas are located in data/. Shell scripts for batch downloading data are located in shell_scripts/. While shell scripts can be run directly in a macOS Terminal, running them in Windows requires Cygwin (and perhaps slight modification of commands). Python syntax examples are located in python/, which may be grow to become a module in the future.
The only true dependency is nctoolbox.
It is recommended to also download Gibbs-SeaWater (GSW) Oceanographic Toolbox. A benefit of ocean_data_tools is that neatly packs data into uniform structs; at which point a user can easily apply custom calculations or functions from other toolboxes such as GSW. See an example.
To access help, run the command doc ocean_data_tools.
- Want to make changes or add a new function? (1) Fork the repository (make your own separate copy), (2) make changes, and (3) open a 'pull request'. Once approved, it can be merged into the master branch. If you wish to chat beforehand about your contribution, open an issue or email lnferris@alum.mit.edu.
- Don't use git often and don't want to remember all the terminal commands? Download GitHub Desktop.
- Find a bug in the code? Open an 'issue' to notify contributors and create an official record.
Before contributing, please see Contents and consider how your function fits into ocean_data_tools and its ethos of structure arrays. At a minimum, functions must be well-documented and address a specific freely-available oceanographic data source which can be accessed by anyone online.
Adding a new function isn't the only way to contribute. Python, Julia, etc. translations of existing Matlab functions are also welcomed!
If you are interested in becoming a formal collaborator (e.g. have direct access and co-manage this repository), please reach out.
argo_build searches the locally-stored Argo profiles matching the specified region & time period and builds a uniform struct
glider_build loads an archived glider survey (downloaded from gliders.ioos.us/erddap) and builds a uniform struct
mocha_build_profiles builds a uniform struct of profiles from the MOCHA Mid-Atlantic Bight climatology
model_build_profiles builds a uniform struct of profiles from HYCOM or Operational Mercator CMEMS GLOBAL_ANALYSIS_FORECAST_PHY_001_024
waves_build builds a uniform struct of timeseries from CMEMS Global Ocean Waves Multi Year product GLOBAL_REANALYSIS_WAV_001_032
whp_cruise_build builds a uniform struct of profiles from GO-SHIP cruise data in WHP-Exchange Format
woa_build_profiles builds a uniform struct of profiles from World Ocean Atlas 2018 Statistical Mean for All Decades, Objectively Analyzed Mean Fields
wod_build builds a uniform struct of profiles from World Ocean Database data
Don't see a function yet for your preferred data source? Email lnferris@alum.mit.edu to request or contribute.
general_depth_subset subsets a uniform struct by depth
general_map plots coordinate locations in a uniform struct, with optional bathymetry contours
general_profiles plots vertical profiles in a uniform struct
general_region_subset subsets a uniform struct by polygon region
general_remove_duplicates removes spatially (or spatially and temporally) non-unique profiles from a uniform struct
general_section plots a data section from a uniform struct
mocha_domain_plot plots a 3-D domain from the MOCHA Mid-Atlantic Bight climatology
mocha_simple_plot plots a 2-D layer from the MOCHA Mid-Atlantic Bight climatology
model_domain_plot plots a 3-D domain from HYCOM or Operational Mercator CMEMS GLOBAL_ANALYSIS_FORECAST_PHY_001_024
model_simple_plot plots a 2-D layer from HYCOM or Operational Mercator CMEMS GLOBAL_ANALYSIS_FORECAST_PHY_001_024
woa_domain_plot plots a 3-D domain from World Ocean Atlas 2018 Statistical Mean for All Decades, Objectively Analyzed Mean Fields
woa_simple_plot plots a 2-D layer from World Ocean Atlas 2018 Statistical Mean for All Decades, Objectively Analyzed Mean Fields
bathymetry_extract extracts a region of Smith & Sandwell Global Topography and outputs as arrays
bathymetry_plot adds bathymetry to 2-D (latitude vs. longitude) or 3-D (latitude vs. longitude vs. depth) data plots
bounding_region finds the rectangular region around a uniform struct and/or list of coordinates to pass as an argument for other bathymetry functions
bathymetry_section adds Smith & Sandwell Global Topography to a section from plot using bathymetry data nearest to specified coordinates
argo_platform_map plots locations of Argo profiles in a uniform struct, coloring markers by platform (individual Argo float)
argo_platform_subset subsets a uniform struct of Argo data to one platform (individual Argo float)
argo_profiles_map plots coordinate locations of Argo profiles in uniform struct argo, using colors corresponding to argo_profiles called on the same struct
argo_profiles plots vertical Argo profiles in uniform struct argo, using colors corresponding to argo_profiles_map called on the same struct
region_select creates coordinate list (which represents vertices of a polygon region) by clicking stations on a plot
transect_select creates a coordinate list (which represents a virtual transect) by clicking stations on a plot
There two types of datasets: those that need to be downloaded manually1 and those that can be accessed remotely2 through OpenDAP (e.g. the data can be accessed directly on the the internet using a url).
Download Argo data directly from GDAC FTP servers using either the Coriolis selection tool, or the US GDAC. See the Argo User's Manual for more information.
Alternatively run shell_scripts/download_argo to download data via File Transfer Protocol.
To get bathymetry data (for bathymetry_dir), download Smith & Sandwell under Global Topography V19.1 in netcdf form (topo_20.1.nc).
Vist gliders.ioos.us/erddap. Click "View a List of All 779 Datasets" or use the "Advanced Search". After choosing a dataset, navigate to the Data Access Form. To get started, select these variables:
Scroll to "File type:". In the drop-down menu, select ".nc". Click "Submit".
The url for MOCHA Mid-Atlantic Bight climatology is embedded. See Rutgers Marine catalog.
HYCOM data may be accessed remotely using OpenDAP. Get the data url by visiting the HYCOM website. For example, click Access Data Here -> GLBv0.08/expt_57.7 (Jun-01-2017 to Sep-30-2017)/ -> Hindcast Data: Jun-01-2017 to Sep-30-2017. Click on the OpenDAP link. Copy the url as and use this as the source in model_build_profiles.
Alteratively, download subsetted HYCOM data using NCSS. Get the data url by visiting the HYCOM website. For example, click Access Data Here -> GLBv0.08/expt_57.7 (Jun-01-2017 to Sep-30-2017)/ -> Hindcast Data: Jun-01-2017 to Sep-30-2017. Click on the NetcdfSubset link. Set constraints and copy the NCSS Request URL at the bottom of the page. Run shell_scripts/download_hycom_lite. To download multiple months or years, run shell_scripts/download_hycom_bulk_daily (partition files by day) or shell_scripts/download_hycom_bulk_monthly (partition files by month). Please use responsibly.
For Mercator, download Copernicus Marine data directly from FTP servers. First make a Copernicus account. Use the selection tool to download GLOBAL_ANALYSIS_FORECAST_PHY_001_024. Alternatively run shell_scripts/download_mercator. Before running the script, follow the instructions for modifying your ~/.netrc file in the comments of the script.
First make a Copernicus account. Use the selection tool to download CMEMS Global Ocean Waves Multi Year product GLOBAL_REANALYSIS_WAV_001_032.
For GO-SHIP data, get CTD data (for ctdo_dir) by choosing a GO-SHIP cruise and downloading the CTD data in whp_netcdf format. More information about whp_netcdf parameters is available here. Get LADCP data (for uv_dir, wke_dir) here. There is information about LACDP processing here.
Functions build the World Ocean Atlas url at maximum resolution based on arguments, but coarser resolutions and seasonal climatologies are available at the NODC website. Note NCEI is scheduled to update data urls in the near future. Functions will be updated as such.
Search the World Ocean Database and select products.
Ferris, L., (2020). ocean_data_tools: A MATLAB toolbox for interacting with bulk freely-available oceanographic data. Journal of Open Source Software, 5(54), 2497. https://doi.org/10.21105/joss.02497