Huge Maps

[hmb1]

@czender
ncks -O --grd_src=SCRIPgrid_1km-merge-10min_HYDRO1K-merge-nomask_c130402.nc --grd_dst=grids/ne512np4_scrip_c20190125.nc

With some tweaking I managed to run the above on e3sm.ess.uci.edu.
Area --grd_src= 3.630634857373313 ( number grid cells 125, 599,447 )
Area --grd_dst= 12.56637061435908 ( number grid cells 14, 155,778 )
Area overlap= 3.63053993 or 99.9973853781% ( number of grid cells 178, 996, 015 )

The number of grid cells from src having missing area greater than 5% was 17, 289
Of these 83 had potential overlaps of 1000.
( nb 1000 is the max number of potential overlap cells from src - This is currently hard coded )

We are dealing with huge numbers of grid cells here - its gonna take a while find the missing area's. I think the best cause of action is to start with ne512 grid and see how that plays with other lower resolution grids.

[czender]

So it's sort of working and sort of not working. You have demonstrated good performance with ne30 grids. Before working further with ne512, please move step-by-step in that direction by verifying good performance with ne120 and then ne240 grids (pentagons). Compare NCO directly to ERWG and TR performance on the same grids if possible using the min/max/rms procedure I sent you. Also, I need a glossary to understand you. What does this mean: "Of these 83 had potential overlaps of 1000."

[hmb1]

@czender
A search of the kdtree with a bounding box from the --src cell of interest returns a list of "potential overlap" cells from --dst ,the intersection algo is then called with the src cell and each cell this list.

So for example when --src grid is very corse and --dst grid is very fine then the number of potential overlaps ( and actual) will be quite high. Or given a polar cap and a RLL grid the number of cells overlapping could be quite big

The size size of this list is currently hard coded.

[czender]

Please update the NCO performance on this grid