Modern Polars - 3 Performance

[giscus[bot]]

Modern Polars - 3  Performance

A side-by-side comparison of the Polars and Pandas libraries.

https://kevinheavey.github.io/modern-polars/performance.html

[pankajp]

With newer polars version, the slowness in 3.4 Polars can be slower than Pandas sometimes, maybe seems to have disappeared, at least in the specific machine (RHEL7) and versions I have checked (python 3.10, pandas 1.5.3, polars 0.16.7, installed from pip binary wheels).
Polars 0.16.7: 1.8s
Pandas 1.5.3: 3.9s

[kevinheavey]

Still slower on my machine :(

[johnsnow09]

You have amazing knowledge of Polars. Would definitely follow your channel on Youtube if you have any. Happy to learn from you :)

[BrettMcS]

I just learned about pivot from Chapter 4 Reshaping and Tidy Data and can confirm that code - mine - written in ignorance of pivot is truly horrendous. I'm now really starting to appreciate Polars. Many thanks.

[joun1128]

the code In section 3.4 achived 1.47 s ± 55.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each). It is likely to show a difference according to polars version or the number of cpu core.
cpu : AMD Ryzen 5 5600X (6 core 12 thread)
python : 3.10.9
polars : 0.19.7

[AdrienDart]

Thanks so much for all of this! Very helpful.
I just wanted to flag that I think we can use udf like this as well: parse_date_pl(pl.col(date_cols))