New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can I trouble you for code review to integrated newer version of parquet4s into delta DSR/DSW? #256
Comments
@MironAtHome Sure thing! |
Most of the changes were to wrap file path expressed as string into .Path imported from latest version of parquet4s parquet implementation. |
I think that this can be helpful: https://mjakubowski84.github.io/parquet4s/docs/migration/ |
Regarding
|
Great thank you for your guidance and help. |
Hey Marcin, pr is ready |
Two comment from me only. One is a minor code change but the other worries me. I see that Delta relies on some ancient version of parquet-hadoop. I wonder if that can be upgraded without any issue. |
Hey Marcin, sorry for long time to turn around, I had to get over case of covid.
here is assertion text, with a few trace rows printed prior: This is affecting DeltaDataReaderSuite.scala I would much appreciate your help to verify this issue. |
Hi @MironAtHome. I do not see the code as your repo is private. |
Marcin, it is so nice to have your comments. |
I took me time to debug those tests. Thousands of dependencies and buggy resource loading of golden tables. So, the issue seems to be with the test data or with the test itself. Tests fail on |
Closing due to inactivity. |
Ok. Let me revisit tests and code. |
Hi! |
Stack trace: |
This line LOL makes a lot of sense. Still, it would be nice to trace and fix, agreed? |
TBH, I haven't been using Windows for many, many years, so it is the first time I have seen such an error :) Thanks for spotting it! |
I am not sure how much effort, just asking if you would be willing and available to look over PR.
It's currently stack at 1.2.1 and it's just too far back. Causes troubles with deps and I really don't like shading if I have to use it as a workaround.
The text was updated successfully, but these errors were encountered: