Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feather format written by PyArrow 7.0.0 cannot be opened in Arquero #279

Open
josesho opened this issue May 20, 2022 · 1 comment
Open

Comments

@josesho
Copy link

josesho commented May 20, 2022

Hi,

with pyarrow v7.0.0:

import seaborn as sns
import pyarrow as arrow
iris = sns.load_dataset("iris")
pf.write_feather(iris, 'iris.feather')

The resultant file is then loaded into arquero in an Observable notebook. The table has the correct columnar structure but the internal data is in a different format.

Screenshot 2022-05-20 at 10 16 57

I think this is related to #270? And hopefully resolved in #277? Hope it can get PRed soon!

PS I use pyarrow to (lightly) munge and convert a 4GB dataset to a smaller 36MB feather file, and I'd like to start building a dashboard with it, using arquero as the analytics engine.

Thanks for all the hard work!

@josesho
Copy link
Author

josesho commented Jun 15, 2022

I see that with arquero>=5.0, this issue is resolved, with the caveat that only uncompressed Feather files can be read in for the time being!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant