Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(slightly more) complex dataset indexing #119

Open
clbarnes opened this issue Jun 26, 2019 · 1 comment
Open

(slightly more) complex dataset indexing #119

clbarnes opened this issue Jun 26, 2019 · 1 comment

Comments

@clbarnes
Copy link
Contributor

clbarnes commented Jun 26, 2019

Inspired by z5py and with an eye on libraries doing similar things, I wrote https://github.com/clbarnes/h5py_like , which has a bunch of abstract base classes for Files, Groups and Datasets. An example where it wraps rust-n5 is here: https://github.com/clbarnes/rust-pyn5/tree/h5pylike/pyn5 . I don't propose rebasing z5py on that, but h5py_like's shape_utils module may be of interest: I started from the z5py implementation but have extended it in a couple of ways.

Specifically, it handles non-1 striding, and newaxes. It does it in a naive way: it lets the underlying library read out the unstrided data, and then strides (including reversing) in numpy if necessary (which is presumably a copy operation). Newaxes and squeezes are then applied by reshaping, which I think just returns a view.

@constantinpape
Copy link
Owner

Specifically, it handles non-1 striding, and newaxes. It does it in a naive way: it lets the underlying library read out the unstrided data, and then strides (including reversing) in numpy if necessary (which is presumably a copy operation).

Cool! Supporting this in z5py directly would also be nice.

It does it in a naive way: it lets the underlying library read out the unstrided data,

I don't think we can do anything more clever here, so this sounds like a good solution.

A PR for strided views and newaxis support would be very welcome ;).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants