Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds a pass at an alternative lazy parquet reader implementation #1484

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

ianoc
Copy link
Collaborator

@ianoc ianoc commented Jan 28, 2016

Needs some more work before merging around cleaning it up, more benchmarks and seeing what does/doesn't need to be in lui.

TBD are real world tests against jobs with manual projection.

@isnotinvain
Copy link
Contributor

One thing I'm working on is seeing if any of the classes that are mostly scala re-writes of parquet classes can either use the original parquet classes, or even better be skipped entirely. Parts of Parquet's class hierarchy can be pretty confusing, I'd rather not mirror that all over again here, especially the classes with non trivial code in them (which have tests in parquet but not here). So for the stuff we need to keep, I'm also working on adding some tests.

@CLAassistant
Copy link

CLAassistant commented Nov 16, 2019

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants