Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(python): sync the compression argument of sink_ipc and read_ipc #15644

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

eitsupi
Copy link
Contributor

@eitsupi eitsupi commented Apr 14, 2024

compression="uncompressed" is allowed for scan_ipc().

@github-actions github-actions bot added internal An internal refactor or improvement python Related to Python Polars labels Apr 14, 2024
@ritchie46
Copy link
Member

This is breaking? And I think we should discuss which direction we want to sync. I like None as uncompressed in python.

@eitsupi
Copy link
Contributor Author

eitsupi commented Apr 14, 2024

This is breaking? And I think we should discuss which direction we want to sync. I like None as uncompressed in python.

No, this is not a breaking change.
None will continue to be allowed. The only change is that the new "uncompressed" will be allowed in sink_ipc().

Edit: Sorry, I made a different change than intended. I will correct it.

Copy link

codecov bot commented Apr 14, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 81.30%. Comparing base (7341aee) to head (0718087).
Report is 2 commits behind head on main.

Files Patch % Lines
py-polars/polars/lazyframe/frame.py 0.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #15644      +/-   ##
==========================================
- Coverage   81.30%   81.30%   -0.01%     
==========================================
  Files        1373     1373              
  Lines      176243   176245       +2     
  Branches     2544     2545       +1     
==========================================
- Hits       143293   143292       -1     
- Misses      32466    32469       +3     
  Partials      484      484              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@stinodego
Copy link
Member

And I think we should discuss which direction we want to sync. I like None as uncompressed in Python.

I don't think using None is a good idea here. None usually indicates unspecified, which means 'use a default' or 'infer the proper value'. Explicitly passing "uncompressed" is much more clear. I think we should at least support passing "uncompressed".

This is already the case for most write functions on the Python side. None is implicitly supported by converting to "uncompressed" internally, which is what this PR is also trying to do, hence no breaking change here.

On the Rust side, ParquetCompression already includes Uncompressed in the Enum. I think that makes sense for the same reason stated above. If you agree, I think we should update IpcCompression and AvroCompression to match.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internal An internal refactor or improvement python Related to Python Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants