Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support additional create_dataset kwargs #55

Open
rly opened this issue Apr 20, 2024 · 0 comments
Open

Support additional create_dataset kwargs #55

rly opened this issue Apr 20, 2024 · 0 comments
Assignees

Comments

@rly
Copy link
Contributor

rly commented Apr 20, 2024

The LINDI group writer create_dataset supports the kwargs chunks, compression, and compression_opts and the compression-related kwargs are allowed only if compression=="gzip". This is the most popular use case.

HDMF supports a few additional kwargs for create_dataset via H5DataIO, e.g., compression that is not gzip, such as through hdf5plugin (see pynwb tutorial), compression_opts, maxshape, fillvalue, shuffle, fletcher32. In practice, fillvalue and shuffle are rarely if ever used. Blosc compression through hdf5plugin is sometimes used because it is usually better than gzip. Maxshape will be used more often, pending an upcoming change hdmf-dev/hdmf#1064. Note that h5py.Dataset supports even more kwargs that HDMF does not currently support.

I started adding support for additional compression algorithms as a nice exercise and will see how far I can get. I'll update here in a few days. This is not high priority but it would be nice to have more complete support for writing/appending datasets via PyNWB to a LINDI file.

@rly rly self-assigned this Apr 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant