Switch to PyO3 0.21's native datetime support, now available with abi3

[itamarst]

Per @stinodego:

PyO3 has support for the Python datetime module:
https://pyo3.rs/main/doc/pyo3/types/struct.pydatetime

However, this was never available under the abi3 feature flag.

So we have a bunch of workarounds in place to materialize/read datetime objects. Specially, we have defined Python functions here:
https://github.com/pola-rs/polars/blob/main/py-polars/polars/_utils/convert.py

...that are called from Rust, for example here:

polars/py-polars/src/conversion/any_value.rs

Lines 61 to 64 in 8ef2e21

	AnyValue::Date(v) => {
	let convert = utils.getattr(intern!(py, "to_py_date")).unwrap();
	convert.call1((v,)).unwrap().into_py(py)
	},

The 0.21 release notes mention:

Extended chrono / datetime conversions, including support for the abi3 feature and the chrono-tz crate

So I am expecting that we can now take advantage of native PyO3 datetime support. And I expect a good performance boost + code cleanup to come from this.

Originally posted by @stinodego in #15215 (comment)

[itamarst]

Gonna do this next.

[itamarst]

In addition to the to_py_date() and friends, there's also logic in the opposite direction that might also be possible to remove as part of this.

[itamarst]

PyO3/pyo3#3266 means that timezone conversions are lossy, the string timezone is converted to a fixed offset in seconds. So might not be able to convert everything immediately.

[ritchie46]

Interesting. That saves a lot of sub-optimal back and forth.

[itamarst]

Digging further, it may be that the PyO3 limitation is tied to version of Python, and upping minimum supported version to 3.9 will fix it. Python 3.8 security EOL is October 31, 2024...

[itamarst]

(Around 5% of Polars downloads are Python 3.8, per https://pypistats.org/packages/polars).

[stinodego]

FYI, so far we have always kept support for Python versions until their end of life, and then dropped support in the next release. Which I think is a good policy.

So if Python 3.8 support is problematic, this would be blocked until November this year.

[ritchie46]

Yes, we will follow python in this regard.

[itamarst]

Ok, will just file a ticket for second pass optimization (#16199).