Merging WarpedVRTs?

[lukas-paulun]

Hi all!

Situation:
I have a number of large (between 100 and 1000 MB) satellite images in my cloud storage. The images are stored as cloud-optimized GeoTIFFs. I would like to create small tiles with a given size and resolution from the satellite images to run the tiles through a machine learning model.

The pre-release version of rasterio.merge.merge() seems to be the perfect solution: I open my satellite images with rasterio.open() and pass the list of open DatasetReaders to rasterio.merge.merge(). There I can also set the bounds and the resolution of the target tile. Due to the cloud-optimized format, only the data that is actually required for the output tile is loaded

Problem:
Not all of the satellite images are in the same CRS. I wanted to use lazy warping to create WarpedVRT objects in the correct CRS while not loading more data than necessary from the cloud because my code is also running in a cloud resource and has very limited memory resources. However, rasterio.merge.merge() does not accept WarpedVRT objects, only DatasetReaders.

What solution do you suggest?

[groutr]

After looking at merge.merge() it seems like it will work on any dataset-like object. WarpedVRT objects behave like Dataset objects (at least as far as merge is concerned). Did passing WarpedVRT objects result in an error for you?

[sgillies]

Yes, WarpedVRT derives from DatasetReaderBase and should have the methods needed by merge. If it doesn't work, let us know.

[lukas-paulun]

Thanks for the replies!

The error I get is this:

File "/home/lukaspaulun/venvs/pleiades_neo_api/lib/python3.11/site-packages/rasterio/merge.py", line 379, in merge
temp_src = src.read(
^^^^^^^^^
File "rasterio/_warp.pyx", line 1283, in rasterio._warp.WarpedVRTReaderBase.read
ValueError: WarpedVRT does not permit boundless reads

But I do need boundless reads because I sometimes have gaps between my images that I just want filled with nodata values.

[sgillies]

@lukas-paulun it's best to make your WarpedVRTs larger so that they overlap without gaps.

[lukas-paulun]

Okay, but merge.merge() still calls read(..., boundless=True) and this will always lead to the error, no matter whether a boundless read is actually necessary or not, right?

[sgillies]

Oh, I see... we'll need a solution for that. Thanks for explaining!