Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use apache arrow for feather/parquet instead of feather directly #185

Open
sebkopf opened this issue Apr 27, 2023 · 0 comments
Open

use apache arrow for feather/parquet instead of feather directly #185

sebkopf opened this issue Apr 27, 2023 · 0 comments

Comments

@sebkopf
Copy link
Contributor

sebkopf commented Apr 27, 2023

https://github.com/apache/arrow/tree/main/r
https://github.com/wesm/feather

see the blog post about move of feather to arrow here:
https://arrow.apache.org/blog/2019/08/08/r-package-on-cran/

generally --> consider switching to feather based iso_save() to automatically provide R and python support (not sure exactly how that would work but will look into it)

Arrow framework:

note that polars (see #186) is already build on arrow framework

big question is whether to export to feather (faster read/write) or parquet (smaller file storage):

@sebkopf sebkopf changed the title use apache arrow for feather instead of feather directly use apache arrow for feather/parquet instead of feather directly May 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant