Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define a common read_h5 function for _h5py + pandas_ and _dask array_ #9

Open
andypbarrett opened this issue Aug 13, 2020 · 5 comments

Comments

@andypbarrett
Copy link
Collaborator

def read_h5(fname, vnames=[]):
    """Read a list of vars [v1, v2, ..] -> 2D."""
    f = h5py.File(fname, 'r')
    return np.column_stack([f[v][()] for v in vnames])

could be used for the pandas and dask array cells. Maybe this could be added to icepyx or offered as part of a separate tool set.

@asteiker
Copy link
Member

asteiker commented Aug 2, 2021

@andypbarrett Has this been suggested to the icepyx project? Is there still value in us pursuing this within IceFlow and/or any of our other tutorial notebooks?

@andypbarrett
Copy link
Collaborator Author

I don't know about icepyx. It has been a while since I have attended.

However, reading H5 files is problematic because the structure is not consistent. Many of the ATL?? files have the data buried in groups or groups, which makes simple general solutions difficult to find.

@betolink and I were discussing using fsspec or something similar to hardcode a recipe to read the ICESat-2 files.

@asteiker
Copy link
Member

asteiker commented Aug 5, 2021

I think your Issue was originally filed in reference to IceFlow. So perhaps this is better left under their backlog versus an addition to our Tutorials repo itself?

@betolink
Copy link
Member

betolink commented Aug 6, 2021

Reading IS2 HDF files using a common data model is beyond IceFlow. Many projects could benefit from these capabilities. I also wonder what's the status with Icepyx. That's probably the best place to implement it.

@asteiker
Copy link
Member

asteiker commented Jan 23, 2024

CRYO-199

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🆕 New
Development

No branches or pull requests

3 participants