Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic when calling fill_null on Date column with float value #16156

Closed
2 tasks done
mkleinbort-ic opened this issue May 10, 2024 · 1 comment · Fixed by #16182
Closed
2 tasks done

Panic when calling fill_null on Date column with float value #16156

mkleinbort-ic opened this issue May 10, 2024 · 1 comment · Fixed by #16182
Assignees
Labels
A-exceptions Area: exception handling A-panic Area: code that results in panic exceptions accepted Ready for implementation P-low Priority: low python Related to Python Polars

Comments

@mkleinbort-ic
Copy link

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl 
import numpy as np 

# This code used to work (at least as of version 0.20.21)
(pl
 .DataFrame({
     'date1': ['2020-01-01', '2020-01-02'],
     'date2': ['2020-01-01', None]
  })
 .cast({'date1':pl.Date, 'date2':pl.Date})
 .filter(
     pl.col('date1') > pl.col('date2').fill_null(-np.inf)
    )
 )

Now raises:

PanicException: implementation error, cannot get ref Int32 from Float64

Log output

No response

Issue description

There has been a minor change in behavior which I could not find documented in the release notes.

Expected behavior

I agree with the error, but a better error message would be better.

Installed versions

--------Version info---------
Polars:               0.20.25
Index type:           UInt32
Platform:             Windows-10-10.0.22621-SP0
Python:               3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)]

----Optional dependencies----
adbc_driver_manager:  0.11.0
cloudpickle:          3.0.0
connectorx:           0.3.3
deltalake:            <not installed>
fastexcel:            0.10.4
fsspec:               2023.12.2
gevent:               <not installed>
hvplot:               <not installed>
matplotlib:           3.8.4
nest_asyncio:         1.6.0
numpy:                1.26.4
openpyxl:             <not installed>
pandas:               2.2.2
pyarrow:              15.0.0
pydantic:             2.6.1
pyiceberg:            0.6.1
pyxlsb:               <not installed>
sqlalchemy:           2.0.30
torch:                2.3.0+cpu
xlsx2csv:             0.8.2
xlsxwriter:           3.2.0
@mkleinbort-ic mkleinbort-ic added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels May 10, 2024
@MarcoGorelli MarcoGorelli added P-low Priority: low A-exceptions Area: exception handling accepted Ready for implementation and removed bug Something isn't working needs triage Awaiting prioritization by a maintainer labels May 10, 2024
@stinodego stinodego removed the accepted Ready for implementation label May 12, 2024
@stinodego
Copy link
Member

stinodego commented May 12, 2024

For clarity: this should not work, but the error message could be better. More minimal repro:

from datetime import date

import polars as pl

df = pl.DataFrame({"date": [date(2022, 1, 1), None]})
result = df.select(pl.col("date").fill_null(1.0))
print(result)
thread '<unnamed>' panicked at crates/polars-core/src/series/mod.rs:938:13:
implementation error, cannot get ref Int32 from Float64
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
  File "/home/stijn/code/polars/py-polars/repro.py", line 6, in <module>
    result = df.select(pl.col("date").fill_null(1.0))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/stijn/code/polars/py-polars/polars/dataframe/frame.py", line 8132, in select
    return self.lazy().select(*exprs, **named_exprs).collect(_eager=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/stijn/code/polars/py-polars/polars/lazyframe/frame.py", line 1816, in collect
    return wrap_df(ldf.collect(callback))
                   ^^^^^^^^^^^^^^^^^^^^^
pyo3_runtime.PanicException: implementation error, cannot get ref Int32 from Float64

For your use case, I would recommend filling with date.min instead.

@stinodego stinodego added the A-panic Area: code that results in panic exceptions label May 12, 2024
@stinodego stinodego changed the title "implementation error, cannot get ref Int32 from Float64" when using .fill_null(np.inf) on a pl.Date column Panic when calling fill_null on Date column with float value May 12, 2024
@ritchie46 ritchie46 self-assigned this May 13, 2024
@c-peters c-peters added the accepted Ready for implementation label May 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-exceptions Area: exception handling A-panic Area: code that results in panic exceptions accepted Ready for implementation P-low Priority: low python Related to Python Polars
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

5 participants