Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when Xee attempts to cast image data to an expected dtype #146

Open
KMarkert opened this issue Feb 28, 2024 · 2 comments
Open

Error when Xee attempts to cast image data to an expected dtype #146

KMarkert opened this issue Feb 28, 2024 · 2 comments
Labels
question Further information is requested

Comments

@KMarkert
Copy link
Contributor

In some rare cases the data extraction process from EE to local ends up with an error: ValueError: When changing to a smaller dtype, its size must be a divisor of the size of original dtype. It is not clear to me why this occurs but obviously something is going on between the return dtype from EE and the expected dtype from xee.

This issue was introduced by #113.

More investigation is need to work on a reproducible example that demonstrates why this occurs and provide a fix.

@naschmitz naschmitz added the question Further information is requested label Mar 5, 2024
@naschmitz
Copy link
Collaborator

How frequently do you see this issue? Do you think it warrants a rollback?

@KMarkert
Copy link
Contributor Author

KMarkert commented Apr 22, 2024

This seems like a pretty isolated error so no rollback necessary in my opinion.

I did some testing and it seems like this was happening for me when there was an image that was all masked was being requested in the same chunk as non-masked images. This can create a mismatch in data types (floats and ints in this case) then the float values from the mask could not be cast as ints.

I was using xee v0.09. Upgrading to latest version (v0.0.12 as time of posting) seems to have fixed the issue and get the data types correct.

But just in case others run into this issue, one fix I tried was to manually set the chunk size where each image in time is requested individually (e.g. ds.chunk({"time": 1, "lon": x_dim, "lat":y_dim})) before requesting the data locally. That way there isn't a mismatch of dytpes in the array that is sent from Earth Engine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants