Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apache Arrow Flight support #551

Open
SemanticBeeng opened this issue Jan 25, 2019 · 0 comments
Open

Apache Arrow Flight support #551

SemanticBeeng opened this issue Jan 25, 2019 · 0 comments

Comments

@SemanticBeeng
Copy link

SemanticBeeng commented Jan 25, 2019

Consider support for Apache Arrow Flight - data as a service paradigm

When combining "big data" / data science applications with microservices there is an impedance in how such big data is to be marshaled (both format and efficiency).

Apache Arrow aims to create a "shared runtime" between JVM, Python and R and used advanced techniques like "zero copy", memory mapping, etc.

This will allow writing distributed, polyglot algorithms (the "business logic" of data science apps)
https://www.slideshare.net/wesm/data-science-without-borders-jupytercon-2017
https://www.youtube.com/watch?v=wdmf1msbtVs&feature=youtu.be&t=790

Apache Arrow Flight aims to extend that performance to services.
https://www.slideshare.net/JacquesNadeau5/apache-arrow-flight-overview

Seems to me that Mu is a great place to introduce that support to the Scala world.

Please advise if this makes sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant