Skip to content

CommonDataModel.jl is a module that defines types common to NetCDF and GRIB data

License

Notifications You must be signed in to change notification settings

JuliaGeo/CommonDataModel.jl

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Build Status codecov.io documentation stable documentation dev

This package contains abstracts type definition for loading and manipulating GRIB, NetCDF, geoTiff and Zarr files. This package aims to follow the Common Data Model and the CF (climate and forecast models) Metadata Conventions.

Format Package read support write support
NetCDF NCDatasets
OPeNDAP NCDatasets -
GRIB GRIBDatasets -
geoTIFF TIFFDatasets -
Zarr ZarrDatasets

Features include:

  • query and edit metadata of arrays and datasets
  • virtually concatenating multiple files along a given dimension
  • create a virtual subset (view) by indices or by values of coordinate variables (CommonDataModel.select, CommonDataModel.@select)
  • group, map and reduce a variable (CommonDataModel.groupby, CommonDataModel.@groupby) and rolling reductions like running means CommonDataModel.rolling)

Here is minimal example for loading GRIB or NetCDF files.

import CommonDataModel as CDM
import SomeDatasets # where SomeDatasets is either GRIBDatasets or NCDatasets

ds = SomeDatasets.Dataset("file_name")

# ntime is the number of time instances
ntime = ds.dim["time"] # or CDM.dims(ds)["time"]

# create an array-like structure v corresponding to variable temperature
v = ds["temperature"]

# load a subset
subdata = v[10:30,30:5:end]

# load all data
data = v[:,:]

# load a global attribute
title = ds.attrib["title"]  # or CDM.attribs(ds)["title"]
close(ds)

Most users would typically import GRIBDatasets and NCDatasets directly and not CommonDataModel. One should import CommonDataModel only to extent the functionality of GRIBDatasets and NCDatasets.

File conversions

By implementing a common interface, GRIB files can be converted to NetCDF files using NCDatasets.write:

using NCDatasets
using GRIBDatasets
using Downloads: download

grib_file = download("https://github.com/JuliaGeo/GRIBDatasets.jl/raw/98356af026ea39a5ec0b5e64e4289105492321f8/test/sample-data/era5-levels-members.grib")
netcdf_file = "test.nc"
NCDataset(netcdf_file,"c") do ds
   NCDatasets.write(ds,GRIBDataset(grib_file))
end

About

CommonDataModel.jl is a module that defines types common to NetCDF and GRIB data

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •