Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rs_cover, rs_download, and rs_rasterize should accept multiple GEOJSON files #187

Open
markmester opened this issue Oct 16, 2019 · 1 comment

Comments

@markmester
Copy link

With the introduction of the batching feature extraction (#148), the inputs to rs_cover, rs_download, and rs_rasterize are likely going to be multiple GEOJSON files. It would be nice if these tools accepted multiple files as inputs. My current workflow is as follows:

CURR_DIR=$(shell pwd)

# Download full extract
download-extract:
	wget --limit-rate=1M $(PBF_DOWNLOAD_FILE) -O $(CURR_DIR)/data/extract.osm.pbf

# Cut out the area we are interested in.
cut-extract:
	docker run --rm -v $(CURR_DIR)/data:/osmium/data asymmetric/osmium-tool extract --bbox $(BBOX) data/extract.osm.pbf --output data/map.osm.pbf

# Extract geometries from an OpenStreetMap base map
extract-geometries:
	docker run -it --rm -v $(CURR_DIR)/data:/data --ipc=host --network=host mapbox/robosat:latest-cpu extract --type building /data/map.osm.pbf /data/buildings.geojson

# Generate all Slippy Map tiles which have buildings in them
GEOJSON = $(CURR_DIR)/data
generate-slippy-maptiles: $(GEOJSON)/*.geojson
	@for file in $^ ; do \
	 	bn=$$(basename $$file);\
		docker run -d -it --rm -v $(CURR_DIR)/data:/data --ipc=host --network=host mapbox/robosat:latest-cpu cover --zoom $(ZOOM) /data/$$bn /data/$$bn.tiles; \
	done
@daniel-j-h
Copy link
Collaborator

rs rasterize already handles batch rasterization into the same tiles:

if os.path.exists(out_path):
prev = np.array(Image.open(out_path))
out = np.maximum(out, prev)

You can run rs rasterize multiple times and if there is a rasterized tile already present (e.g. previous batch was rasterizing a building on the left, this tile wants to rasterize a building on the right in the same tile) then we load it up and rasterize the current batch onto it.

This allows for simple and easy batch rasterization with a single rs rasterize command.

For context see #25


The other tools you can call in a bash for loop; not sure if we should explicitly complicate the logic here. But I understand it's no longer as easy as running multiple commands after each other.

If you want to implement this, I'm happy to guide you along and review pull requests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants