Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raster files opened with rioxarray are not closed successfully. #701

Open
riftia opened this issue Oct 9, 2023 · 0 comments
Open

Raster files opened with rioxarray are not closed successfully. #701

riftia opened this issue Oct 9, 2023 · 0 comments
Labels
bug Something isn't working

Comments

@riftia
Copy link

riftia commented Oct 9, 2023

Consider the following workflow:

import rioxarray as rxr

raster_files = glob.glob("/path/to/files") # 50 raster files

for f in raster_files:
    with rxr.open_rasterio(f) as ds:
        # perform some operations and write out a new raster
        ds_max = ds.max(dim='band')
        ds_max.rio.to_raster("/path/to/output/file")

Running this code causes the disk throughput and I/O to rise exponentially after about 20 files have been iterated over, leading to the system freezing. Since the files are being opened within a context manager, they should automatically close.

I tried recreating the same workflow using xarray, and faced no problems.

import xarray as xr

raster_files = glob.glob("/path/to/files") # 50 raster files

for f in raster_files:
    with xr.open_dataset(f) as ds:
        # perform some operations and write out a new raster
        ds_max = ds.max(dim='band')
        ds_max['band_data'].rio.to_raster("/path/to/output/file")

The above code snippet works just fine.

I also tried explicitly calling ds.close() for rioxarray, but the problem still perists.

@riftia riftia added the bug Something isn't working label Oct 9, 2023
@snowman2 snowman2 added this to the 0.15.1 milestone Nov 9, 2023
@snowman2 snowman2 modified the milestones: 0.15.1, 0.15.2 Apr 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants