You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently SWALS writes multiple output files for each domain:
A netcdf with gridded outputs
A netcdf with gauge outputs
Some text metadata
For large multidomains with hundreds of domains, this can lead to > 1000 files for a single model run.
This can be a problem, e.g., if running hundreds of scenarios then we start approaching file-count limits on some supercomputers (e.g. NCI).
An alternative is to use netcdf groups to combine gauge/grid outputs from each domain in a single netcdf file.
This would require updates to SWALS including the post-processing scripts.
A potential downside is that netcdf-groups might not play nicely with some other tools (like ncview). So ideally the netcdf-group format could be implemented as a compile-time option, with post-processing routines seamlessly working with either format.
The text was updated successfully, but these errors were encountered:
This version iteratively adds netcdf files into groups, while using group names to reflect the directory structure. Run from the multidomain directory.
for i in RUN*/Grid*.nc; do echo $i; ncecat -4 --deflate 4 --no_tmp_fl -A -M -G $i --gag $i -o test7.nc; done
We can also put grids and gauges in the one file
for i in RUN*/G*.nc; do echo $i; ncecat -4 --deflate 4 --no_tmp_fl -A -M -G $i --gag $i -o test7.nc; done
Currently SWALS writes multiple output files for each domain:
For large multidomains with hundreds of domains, this can lead to > 1000 files for a single model run.
An alternative is to use netcdf groups to combine gauge/grid outputs from each domain in a single netcdf file.
A potential downside is that netcdf-groups might not play nicely with some other tools (like ncview). So ideally the netcdf-group format could be implemented as a compile-time option, with post-processing routines seamlessly working with either format.
The text was updated successfully, but these errors were encountered: