Fixed pd.infer_freq incompatible with Series["timestamp[s][pyarrow]"]
#58404
Conversation
randolf-scholz
changed the title
pd.infer_freq incompatible with Series["timestamp[s][pyarrow]"]
| ) | ||
| index = values | ||
| if isinstance(index, ABCSeries) and not ( | ||
| is_datetime64_any_dtype(index) or index.dtype == object |
There was a problem hiding this comment.
Can you check dtype.kind == "mM" on ArrowDtype instead?
There was a problem hiding this comment.
I am a bit surprised by this comment, isn't pandas.api.types.is_ supposed to be a single source of truth of what is considered a certain data type? How do you remember what are all the conditions to check otherwise?
There was a problem hiding this comment.
For internal usages, checking dtype.kind is more performant than api.types functions so that is preferred
There was a problem hiding this comment.
Ah I guess I accidentally removed checking for timedelta. Although no tests failed. I am adding some tests for timedeltas as well.
There was a problem hiding this comment.
There is another bug: when trying with Index["duration[s][pyarrow]"], it complains about the property asi8 missing. asi8 is denoted as deprecated on general index types. Moreover, the FrequencyInferer assumes numpy dtype and makes use of internals (like index._data._ndarray)
the easiest workaround would be to cast to numpy, but I guess we want to avoid that and keep dtype as pyarrow inside the ops.
|
This is going to convert to a (non-pyarrow) DatetimeIndex under the hood, which will incur a silent perf penalty. Better to just tell users to convert themselves? |
pd.infer_freqincompatible withSeries["timestamp[s][pyarrow]"]. #58403doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.