Aggregation area limits: convert deg2 to grid cells or km2?

[trevorgerhardt]

A user recently attempted to create an aggregation area that exceeded our 2 deg² limit. Found here:

r5/src/main/java/com/conveyal/r5/analyst/Grid.java

Lines 98 to 99 in dbcc567

	/** Maximum area allowed for features in a shapefile upload */
	private static final double MAX_FEATURE_AREA_SQ_DEG = 2;

Historically, this limit was in place due to high CPU utilization when rasterizing large files. Is that limitation still justified? Are there other reasons why we were limiting to 2deg²? Could we safely increase this limit?

While considering this, I also wondered why this seems to be the only limitation that we have in deg². It may have been used due in that location due to the ease in checking size in those units without a conversion. But users don't typically know what 1 deg² converts to in km². Or how that lines up with grid cells.

For example, Grids are constructed with WebMercatorExtents, which also limit the size, but by checking the total number of grid cells (currently limited to 5,000,000).

Additionally, GeometryUtils limits bounds size by square kilometers. Setting 975,000 km² as the maximum (and explicitly noting that it is large enough for California). The limit corresponds to approximately 16 million grid cells at z9. Used here:

r5/src/main/java/com/conveyal/r5/common/GeometryUtils.java

Lines 164 to 173 in dbcc567

	public static void checkWgsEnvelopeSize (Envelope envelope, String thingBeingChecked) {
	checkWgsEnvelopeRange(envelope);
	checkWgsEnvelopeAntimeridian(envelope, thingBeingChecked);
	if (roughWgsEnvelopeArea(envelope) > MAX_BOUNDING_BOX_AREA_SQ_KM) {
	throw new IllegalArgumentException(String.format(
	"Geographic extent of %s (%.0f km2) exceeds limit of %.0f km2.",
	thingBeingChecked, roughWgsEnvelopeArea(envelope), MAX_BOUNDING_BOX_AREA_SQ_KM
	));
	}
	}

This GeometryUtils method is used to check the bounds of OSM files, GTFS files, data sources, and opportunity datasets (through Grid.java).

Should we consolidate/standardize our geographic limits to either km² or grid cell count? Both in terms of the units that we use and the limitations themselves.

[geodarcy]

Just a comment from a user. While you mention that the current limit is big enough for California, I live in Canada. I'm trying to use r5 through an R library. With a previous version of r5, I successfully built a network for all of Canada, but I mostly need to build networks at the provincial level. With the geographic limit, I cannot build a network for most of the provinces. Even if I need to build networks at the city level, due to IT limitations at work, geofabrik.de is the only place to obtain OSM data and the province is the lowest level that can be downloaded. I would really appreciate the spatial limit being removed or greatly increased. (Also, at one point I noticed a limit against OSM data above 85 degrees north. Canada has valid OSM data north of 85 degrees)