Python Polars 0.20.9

🚀 Performance improvements

• use owned arithmetic in horizontal_sum (#14525)

✨ Enhancements

• Add writable flag to DataFrame.to_numpy (#14520)
• flatten aliases (#14512)
• Make formatting more consistent in DOT graphs (#14486)
• add flush operator to streaming operators (#14500)

🐞 Bug fixes

• ensure the streaming dispatcher can replace placeholders in unions (#14537)
• Ensure series are contiguous prior to transpose (#14527)
• write csv header if necessary when finishing sinks (#14518)
• fix logical dtypes in take_chunked (#14517)
• fix binary-offset row-encode (#14514)
• race conditions in OOC writing (#14510)
• Remove is_numeric check on Series.std/var (#14493)
• Error on invalid schema input in DataFrame constructor (#14483)

📖 Documentation

• Fix docstring example for Config.save_to_file (#14533)
• Fix infer_schema_length param description (#14233)
• Clean up grammar and capitalization in README.md (#14488)
• Add examples for Series.bin.ends_with, Series.bin.starts_with, Series.bin.decode, Series.bin.encode. (#14478)

🛠️ Other improvements

• add code coverage CI (#14532)
• Re-enable streaming OOC tests (#14522)
• Use constant for checking Sphinx building (#14502)

Thank you to all our contributors for making this release possible!
@FBruzzesi, @NedJWestern, @c-peters, @dannyfriar, @i-aki-y, @jdanford, @mbuhidar, @mcrumiller, @ritchie46, @stinodego and @taki-mekhalfa

Python Polars 0.20.8

🏆 Highlights

• Implemented tree formatting for LogicalPlan (#14221)

⚠️ Deprecations

• Deprecate positional args in pivot to prepare new functionality (#14428)

🚀 Performance improvements

• Combine small chunks in sinks for streaming pipelines (#14346)
• reduce heap allocs in expression/logical-plan iteration (#14440)
• simplify and speed up cum_sum and cum_prod (#14409)
• simplify negated predicates to improve row groups skipping (#14370)

✨ Enhancements

• Increase verbosity of duplicate column error message (#11899)
• change print to warn in reading csv from python file like object (#14469)
• Raise if pivot would introduce duplicate column names (#14431)
• apply negate in simplify expression pass (#14436)
• restrict more cloud interop to semaphore budget (#14435)
• Implement min/max for categorical dtype (#14112)
• Hide polars.testing.* in pytest stack traces (#14399)
• expose numpy view to integer types (#14405)
• Allow column name input in clip (#14410)
• add boolean rle decoding for parquet (#14403)
• Allow brackets in SQL join conditions (#14263)
• Implemented tree formatting for LogicalPlan (#14221)
• Implement mean_horizontal expression (#14369)
• support decimal comparison (#14338)
• Implements arr.shift (#14298)
• Implements list.n_unique (#14306)
• Do not panic when casting from an empty Series to pl.Decimal (#14330)
• unset WRITEABLE flag in zero-copy output (#14283)
• Support Categorical/Enum in Series.to_numpy (#14275)
• add parametric testing support for the Array dtype (#14265)

🐞 Bug fixes

• don't gc after variadic buffers are written (#14473)
• Increase verbosity of duplicate column error message (#11899)
• Return appropriate data type for duration mean and median (#14376)
• change print to warn in reading csv from python file like object (#14469)
• regression in out-of-core group-by by new string-type (#14464)
• DataFrame.pivot was returning incorrect results when multiple columns were passed to index and one of them was Struct (#14438)
• remove literal Series from projection state (#14437)
• pivot was producing incorrect results when (single) index was Struct (#14308)
• Error on some invalid clip inputs (#14416)
• Series.hist panicking on empty/all-null (#14407)
• rechunk series when apply_lambda (#14406)
• Raise if invalid strategy is passed to map_elements (#14397)
• Require exact checking for Decimals in assertion utils (#14357)
• fix ufunc for unlimited column args (#14328)
• Handle chunked Series in Series.to_numpy (#14341)
• Remove duplicated content in error messages (#8107)
• Fix set_operation if the input is sliced and be broadcast (#14303)
• Wrap par_iter in list.to_struct by POOL.install (#14304)
• Do not panic when casting from an empty Series to pl.Decimal (#14330)
• Preserve name when casting to Enum (#14320)
• list.get does not work on list of decimals (#14276)
• relax precision when up scaling (#14270)
• Allow format object series with registry (#14272)

📖 Documentation

• Update read_database docstring note about getting the connection URI string for sqlalchemy (#14461)
• Fix typo in plugins section (#14402)
• Add debugging section to contributing docs (#10576)
• Define what a 'character' means in slice / len_chars (#14395)
• Clarify behavior of DataFrame.rows_by_key (#14149)
• Fix some typos (#14394)
• Realign file structure of user guide (#14360)
• Rust examples for data structures in user guide (#14339)
• Add deprecation period policy example for post-1.0.0 (#14184)
• Add example for Series.bin.contains (#14297)
• Small clarifications in the contributing guide (#14310)
• Fix capitalization of user guide references (#14291)
• Fix explode docstring mentioning String types (#14285)
• Update deltalake docstrings to new link (#14282)

🛠️ Other improvements

• Ignore unclosed file warnings for now (#14467)
• Raise better error in import timings test (#14441)
• Refactor arg_min/max test case (#14439)
• Skip some OOC tests that fail randomly in the CI (#14434)
• Bump release drafter to v6 (#14429)
• Set specific temp dir for OOC tests (#14420)
• Bump setup-graphviz action to v2 (#14418)
• Minor test refactor (#14404)
• Update make clean command (#14408)
• Internal rename of _or to or_ in PyO3 (same for _xor/_and) (#14393)
• Minor refactor of DataFrame.to_numpy structured code (#14348)
• Update Series.to_numpy to handle Decimal/Time types in Rust (#14296)
• Add test for Series.to_numpy with timezones (#14337)
• Bump ruff version to 0.2.0 (#14294)
• Temporarily fix failing deltalake test (#14288)
• remove dataframe consortium standard api entrypoint (#14279)

Thank you to all our contributors for making this release possible!
@BGR360, @CaselIT, @MarcoGorelli, @migi, @NedJWestern, @Vincenthays, @alexander-beedie, @deanm0000, @dependabot, @dependabot[bot], @engdoreis, @flisky, @grinya007, @itamarst, @janosh, @kalekundert, @lukemanley, @mbuhidar, @mcrumiller, @petrosbar, @r-brink, @rben01, @reswqa, @ritchie46, @stinodego, @taki-mekhalfa and @thomasfrederikhoeck

Python Polars 0.20.7

⚠️ Deprecations

• Rename threadpool_size to thread_pool_size (#14236)

🚀 Performance improvements

• prune parquet row groups when is_not_null is used (#14260)
• Avoid unnecessary copies in Series.to_numpy for boolean/temporal types (#14261)
• use is_between to skip parquet row groups (#14244)
• Use a compression API that is designed for this use case (#11699) (#14194)
• Use UnitVec in polars-plan traversal (#14199)
• use UnitVec in streaming joins (#14197)
• improve ChunkId (#14175)
• improve iteration performance (#14126)
• elide unneeded work in window? (#14108)
• run window functions more in parallel (#14095)
• improve skip row group using statistics condition (#14056)

✨ Enhancements

• add u8/i8/u16/i16 parsers to CSV reader (#14241)
• move F-order data in and out of numpy to polars zero copy (#14259)
• read arrow-c-interface without requiring pyarrow (#14254)
• Implements list.gather_every (#14253)
• Implements prefix/suffix_fields (#14251)
• Change Series.to_numpy to return f64 for Int32/UInt32 Series with nulls instead of f32 (#14240)
• Polish decimal arithmetic (#14172)
• improved read_excel format detection, and support for excel 97-2004 workbooks (#14234)
• Introduce arr.to_struct (#14202)
• Supports map fields name of struct (#14203)
• make IdxVec generic as UnitVec (#14196)
• add new arithmetic kernels (#14026)
• Supports unique and hash_rows for null column (#14111)
• Implement arithmetic operations for Null columns (#14107)
• support pd.Index in from_pandas and elsewhere (#14087)
• Allow renaming expressions with keyword syntax in group_by (#14071)
• raise more informative error message if someone lands on Expr.__bool__ (#14067)
• Adapt extend_constant to function expr architecture and expressify it (#14058)
• add integer negation (#14049)
• list & array measures of dispersion (#13245)
• gc binview when writing ipc (#14035)
• When calling convert_time_zone on time-zone-naive datetime, convert as if converting from UTC (#13960)

🐞 Bug fixes

• deduplicate recursive growables (#14264)
• Fix glimpse overload signature (#14258)
• allow set operations on list of categoricals (#14110)
• any/all_horizontal with single input has incorrect type (#14256)
• load numpy array with np array values #14237 (#14238)
• Make Series.to_numpy on booleans without nulls return bool type (#14239)
• fix ufunc in agg (change __ufunc_array__ so it uses is_elementwise=True parameter) (#14135)
• Fix join validation for String types (#14229)
• enable windows test coverage for read_excel "calamine" (fastexcel) engine (#14171)
• make csv parser more robust to edge cases (#14210)
• Fix for set_operations of binary dtype (#14152)
• fix read_csv date/datetime inference and parsing (#14113)
• don't see files as hive partitions (#14128)
• allow eval on list of categoricals (#14132)
• Forbid casting from Date to Time and vice versa (#14127)
• preserve old naming convention for multi-value pivot (this will change in 1.0 to no longer redundantly have the column name in the middle) (#14120)
• Implements gt/lt cmp for null dtype (#14119)
• ignore comments at beginning of csv if schema provided (#14115)
• fix pivot when multiple columns are passed. Output is now aligned with what tidyverse / pandas.pivot_table would do (#14048)
• multiple read_excel updates (#14039)
• some temporal conversion errors for datetimes earlier than 1970-01-01 (#14050)
• Preserve name when casting from categorical (#14085)
• respect Object dtype designation (#14072)
• fix cse bug when window function is nested (#14070)
• Fix melt panic when there are no value vars (#14057)
• json_encode should respect the logical type (#14063)
• improve skip row group using statistics condition (#14056)
• Raise for .dt.epoch and .dt.timestamp for Duration dtype (#13962)
• handle SliceSink with empty data (#14025)
• Allow Series.to_pandas for categorical types (#14028)
• correct field type schema inference (using read_csv) (#14042)
• Use int formatter for unsigned ints (#14043)

📖 Documentation

• fix code block in user-guide/lazy/schemas (#14228)
• Add visualization page to user guide (#13052)
• Fix typo in contributing guide (#14181)
• Small improvements Ecosystem page (#14176)
• fix code blocks in user-guide/concepts/data-structures (#14146)
• Document that Kleene logic is followed in any_horizontal and all_horizontal (#14148)
• Fix description of return_dtype parameter for map_elements and map_batches (#14114)
• Fix bullet point formatting in CI contributing guide (#14117)
• Add documentation on replacement strings to str.replace and str.replace_all (#13382)
• Replace alternatives page with more objective comparison (#13784)
• Note that only one name operation is allowed per expression (#14075)
• Improve deprecation message of dtype_if_empty param (#14068)
• fix more docstring bullet points (#14065)

🛠️ Other improvements

• Reorganize NumPy interop tests (#14257)
• additional dataframe test coverage (#14243)
• Remove *args in Series.to_numpy (#14248)
• Move metadata utils to meta module (#14230)
• remove unused method DataFrame._from_dicts (#14212)
• make gather_chunked completely generic (#14195)
• Add .cargo directory to .gitignore (#14191)
• take_chunked to polars-ops (#14185)
• Issue a warning when running doctests on Python 3.11 or lower (#14187)
• Run cargo update (#14160)
• merge take kernels (#14137)
• improve From<Ca> -> Vec (#14123)
• hoist boolean -> string cast (#14122)
• remove unused argument (#14014)

Thank you to all our contributors for making this release possible!
@JulianCologne, @MarcoGorelli, @Vincenthays, @Wainberg, @alexander-beedie, @apcamargo, @braaannigan, @c-peters, @deanm0000, @dependabot, @dependabot[bot], @dpinol, @edavisau, @eitsupi, @flisky, @grinya007, @ion-elgreco, @itamarst, @lukemanley, @mcrumiller, @orlp, @r-brink, @reswqa, @ritchie46, @stinodego and @taki-mekhalfa

Rust Polars 0.37.0

🏆 Highlights

• new implementation for String/Binary type. (#13748)

💥 Breaking changes

• Remove DatetimeChunked::convert_time_zone (#14046)
• Rename LiteralValue::to_anyvalue to LiteralValue::to_any_value (#14033)
• Rename drop_columns to drop (#13754)
• Rename pl.count() to pl.len() (#13719)
• Rename row_count_name/row_count_offset parameters in IO functions to row_index_* (#13563)
• Rename with_row_count to with_row_index (#13494)

🚀 Performance improvements

• prune parquet row groups when is_not_null is used (#14260)
• use is_between to skip parquet row groups (#14244)
• Use a compression API that is designed for this use case (#11699) (#14194)
• Use UnitVec in polars-plan traversal (#14199)
• use UnitVec in streaming joins (#14197)
• improve ChunkId (#14175)
• improve iteration performance (#14126)
• elide unneeded work in window? (#14108)
• run window functions more in parallel (#14095)
• improve skip row group using statistics condition (#14056)
• improve string/binary reverse performance (#14016)
• optimize DataFrame.describe by presorting columns (#13822)
• elide redundant bound checks. (#13909)
• speedup boolean filter (#13905)
• speedup binview filter (#13902)
• improve binview filter (#13878)
• apply string view GC more conservatively (#13850)
• add optimized BinaryViewArray comparison kernels (#13839)
• lazy cache binview bytes len (#13830)
• fast-path for eager int_range (#13811)
• Optimize arr.sum for inner non-null bool (#13800)
• directly embed data ptr in Buffer (#13744)
• elide parallelism restriction on generic rolling expressions (#13662)
• ensure time groups are parallelized (#13660)
• do not eagerly compute bitcount (#13562)
• optimise SQL engine string concat (#13499)
• remove lifetime requirement from CategoricalChunkedBuilder (#13319)

✨ Enhancements

• add u8/i8/u16/i16 parsers to CSV reader (#14241)
• Implements list.gather_every (#14253)
• Implements prefix/suffix_fields (#14251)
• Polish decimal arithmetic (#14172)
• Introduce arr.to_struct (#14202)
• Supports map fields name of struct (#14203)
• make IdxVec generic as UnitVec (#14196)
• add new arithmetic kernels (#14026)
• Supports unique and hash_rows for null column (#14111)
• Implement arithmetic operations for Null columns (#14107)
• Add strict/non-strict construction of Boolean/Binary series (#14073)
• Improve Series::from_any_values logic (#14052)
• Adapt extend_constant to function expr architecture and expressify it (#14058)
• add integer negation (#14049)
• list & array measures of dispersion (#13245)
• gc binview when writing ipc (#14035)
• When calling convert_time_zone on time-zone-naive datetime, convert as if converting from UTC (#13960)
• DataFrame supports explode by array column (#13958)
• improve binary formatting (#13981)
• preserve Enum information when going to IPC (#13943)
• support kwargs in plugin 'field' functions and raise error on unsupported binview layout (#13944)
• support cast decimal to utf8 (#13829)
• add SQL support for timestamp precision modifier (#13936)
• support negative indexing and expressions for LEFT, RIGHT and SUBSTR SQL string funcs (#13888)
• Introduce explode for ArrayNameSpace (#13923)
• raise better error message for .dt.time on Date column (#13932)
• List set_operations supports float (#13920)
• Add ignore_nulls for arr.join (#13919)
• register 'set_sorted' as batch/elementwise (#13896)
• move Enum/Categorical categories to binview (#13882)
• Add ignore_nulls for list.join (#13701)
• Add ignore_nulls for pl.concat_str (#13877)
• fix parquet for binview (#13873)
• support mmap for binview in OOC (#13872)
• implement ffi for binview (#13871)
• Support zero fill null strategy for binary and string columns (#13869)
• Implement/fix unary minus operator -pl.col(...) (#13776)
• extend SQL EXTRACT with "century", "millennium", and "timezone" parts (#13634)
• fix binview ipc format (#13842)
• add SQL support for numeric and/or decimal types (#13739)
• improve panic message (#13836)
• Expressify str.zfill (#13790)
• new implementation for String/Binary type. (#13748)
• Add nulls_last for Series.sort (#13794)
• Impl count_matches for array namespace (#13675)
• Add nulls_last for list/array.sort (#13795)
• Rename drop_columns to drop (#13754)
• convert fixed-offset timezones to respective Etc timezone from time zone database (#13738)
• Expressify str.slice (#13747)
• implement binview for polars-row (#13736)
• implement binview for polars-json (#13737)
• add architecture for polars-flavored IPC (#13734)
• implement binview comparison kernels (#13715)
• raise default frame/series repr height from 8 to 10 (#13699)
• write parquet ColumnOrder (#13672)
• Impl contains for ArrayNameSpace (#13638)
• improve rolling() expression formatting (#13657)
• Implement is_between in Rust (#11945)
• Expressify pattern of str.extract (#13607)
• Impl join for ArrayNameSpace (#13586)
• add SQL engine support for string cast to json (#13624)
• add SQL engine support for EXTRACT and DATE_PART (#13603)
• add BinaryView to parquet writer/reader. (#13489)
• add SQL engine support for POSITION and STRPOS (#13585)
• is_in support for array dtype (#13559)
• add new str.find expression, returning the index of a regex pattern or literal substring (#13561)
• add SQL engine support for LIKE and ILIKE pattern matching (#13522)
• improve hive partition pruning (#13358) (#13426)
• don't rechunk by default in lazy scans (#13518)
• Add cum_count expression function (#13478)
• add SQL engine support for IF control flow function (#13491)
• add SQL engine support for MOD function (#13502)
• return datetime for datetime mean & median (#13417)
• add SQL engine support for CONCAT_WS string function (#13483)
• BinaryView/Utf8View IPC support (#13464)
• Implement wasm Pool::scope (#13476)
• add SQL engine support for RIGHT and REVERSE string functions (#13461)
• implement BinaryView and Utf8View in polars-arrow (#13243)
• add SQL engine support for variadic string CONCAT function (#13428)
• add support for AND in SQL join-clause context (#13242)
• Impl ordering ops for array namespace (#13414)
• add SQL engine support for REPLACE string function (#13431)
• add SQL engine support for SIGN function (#13429)
• add SQL engine support for IFNULL function (#13432)
• additional SQL support for bytes, bit, and hex literals (#13389)

🐞 Bug fixes

• deduplicate recursive growables (#14264)
• Fix glimpse overload signature (#14258)
• allow set operations on list of categoricals (#14110)
• any/all_horizontal with single input has incorrect type (#14256)
• load numpy array with np array values #14237 (#14238)
• Fix join validation for String types (#14229)
• make csv parser more robust to edge cases (#14210)
• Fix for set_operations of binary dtype (#14152)
• fix read_csv date/datetime inference and parsing (#14113)
• don't see files as hive partitions (#14128)
• allow eval on list of categoricals (#14132)
• add missing conditional compile flag for StringFunction::Find (#14129)
• Forbid casting from Date to Time and vice versa (#14127)
• preserve old naming convention for multi-value pivot (this will change in 1.0 to no longer redundantly have the column name in the middle) (#14120)
• Implements gt/lt cmp for null dtype (#14119)
• ignore comments at beginning of csv if schema provided (#14115)
• fix pivot when multiple columns are passed. Output is now aligned with what tidyverse / pandas.pivot_table would do (#14048)
• some temporal conversion errors for datetimes earlier than 1970-01-01 (#14050)
• Preserve name when casting from categorical (#14085)
• fix cse bug when window function is nested (#14070)
• Fix melt panic when there are no value vars (#14057)
• json_encode should respect the logical type (#14063)
• improve skip row group using statistics condition (#14056)
• Raise for .dt.epoch and .dt.timestamp for Duration dtype (#13962)
• handle SliceSink with empty data (#14025)
• correct field type schema inference (using read_csv) (#14042)
• Map AnyValue::Null to datatype Null (#14045)
• Use int formatter for unsigned ints (#14043)
• quick fix for multiple chunks binary reverse (#14024)
• count matches on list categorical (#14021)
• list.min/max with empty and/or None elements (#14018)
• allow get access to list of categoricals (#14015)
• Fix casting from categorical to numeric (#13957)
• read_csv preserve whitespace and newlines (#13934)
• append decimal with different scale (#13977)
• Allow casting integer types to Enum (#13955)
• arg_min/max on categoricals should respect ordering (#13998)
• serialize decimal type (#13997)
• check input type for arr/list.contains (#13959)
• Allow dtype merge when inner dtype is enum (#13938)
• recurse less in streaming shared sinks (#13930)
• ensure order is preserved if streaming from different sources (#13922)
• Fix is_not_null for Struct columns (#13921)
• make 100 * pl.col(pl.Boolean).mean() work (#13725)
• allow extract of numeric from str AnyValue (#13865)
• single-element .dt.time() and .dt.date() should always preserve sortedness (#13808)
• prune emtpy chunks before set operations (#13898)
• treat null columns as zero in sum_horizontal (#13880)
• include null count in rolling window validity with min_periods (#13863)
• don't return NaN as free memory fraction (#13860)
• parquet hybrid RLE encoding did not always align to bit width (#13883)
• Add ignore_nulls for list.join (#13701)
• .dt.time() was panicking for datetimes prior to unix epoch (#13812)
• Correct err message of check_map_output_len (#13854)
• allow list creation of decimals (#13851)
  ...

Python Polars 0.20.6

🏆 Highlights

• new implementation for String/Binary type. (#13748)

⚠️ Deprecations

• Deprecate dtype_if_empty parameter for Series constructor (#13976)

🚀 Performance improvements

• improve string/binary reverse performance (#14016)
• add "calamine" support to read_excel, using fastexcel (~8-10x speedup) (#14000)
• optimize DataFrame.describe by presorting columns (#13822)
• elide redundant bound checks. (#13909)
• speedup boolean filter (#13905)
• speedup binview filter (#13902)
• allow python threads in read_ functions (#13886)
• improve binview filter (#13878)
• apply string view GC more conservatively (#13850)
• add optimized BinaryViewArray comparison kernels (#13839)
• lazy cache binview bytes len (#13830)
• fast-path for eager int_range (#13811)
• Optimize arr.sum for inner non-null bool (#13800)

✨ Enhancements

• Add UnstableWarning for unstable functionality (#13948)
• DataFrame supports explode by array column (#13958)
• add "calamine" support to read_excel, using fastexcel (~8-10x speedup) (#14000)
• improve binary formatting (#13981)
• preserve Enum information when going to IPC (#13943)
• support calling describe on a LazyFrame (#13982)
• support kwargs in plugin 'field' functions and raise error on unsupported binview layout (#13944)
• support cast decimal to utf8 (#13829)
• add SQL support for timestamp precision modifier (#13936)
• support negative indexing and expressions for LEFT, RIGHT and SUBSTR SQL string funcs (#13888)
• Introduce explode for ArrayNameSpace (#13923)
• unify Series/DataFrame describe code (#13720)
• raise better error message for .dt.time on Date column (#13932)
• List set_operations supports float (#13920)
• Add ignore_nulls for arr.join (#13919)
• register 'set_sorted' as batch/elementwise (#13896)
• move Enum/Categorical categories to binview (#13882)
• Add ignore_nulls for list.join (#13701)
• Add ignore_nulls for pl.concat_str (#13877)
• Align int_range and int_ranges signatures (#13867)
• fix parquet for binview (#13873)
• support mmap for binview in OOC (#13872)
• implement ffi for binview (#13871)
• Support zero fill null strategy for binary and string columns (#13869)
• allow df.rename and lf.rename to take a renaming function (#13708)
• Implement/fix unary minus operator -pl.col(...) (#13776)
• extend SQL EXTRACT with "century", "millennium", and "timezone" parts (#13634)
• fix binview ipc format (#13842)
• add SQL support for numeric and/or decimal types (#13739)
• improve panic message (#13836)
• Expressify str.zfill (#13790)
• new implementation for String/Binary type. (#13748)
• Add typing to hvplot plot namespace (#13813)
• Add nulls_last for Series.sort (#13794)
• allow ftp URLs, improve URL check (#13781)

🐞 Bug fixes

• count matches on list categorical (#14021)
• list.min/max with empty and/or None elements (#14018)
• Make to_pandas() work for Dataframe and Series with dtype Object (#13910)
• raise for pl.concat(how="align") when no columns are shared between frames (#13941)
• Fix casting from categorical to numeric (#13957)
• read_csv preserve whitespace and newlines (#13934)
• omit implicit 'site' from import-timing test (#14009)
• append decimal with different scale (#13977)
• Use date_as_object=False as default for Series.to_pandas (just like DataFrame.to_pandas) (#13984)
• serialize decimal type (#13997)
• check input type for arr/list.contains (#13959)
• Fix max_colname_length formatting in glimpse() (#13969)
• Allow dtype merge when inner dtype is enum (#13938)
• recurse less in streaming shared sinks (#13930)
• ensure order is preserved if streaming from different sources (#13922)
• Fix is_not_null for Struct columns (#13921)
• convert object-dtyped NumPy str/bytes arrays to pl.String/pl.Binary instead of pl.Object (#13712)
• allow extract of numeric from str AnyValue (#13865)
• single-element .dt.time() and .dt.date() should always preserve sortedness (#13808)
• prune emtpy chunks before set operations (#13898)
• treat null columns as zero in sum_horizontal (#13880)
• include null count in rolling window validity with min_periods (#13863)
• Fix interchange protocol for new String type (#13881)
• parquet hybrid RLE encoding did not always align to bit width (#13883)
• Add ignore_nulls for list.join (#13701)
• .dt.time() was panicking for datetimes prior to unix epoch (#13812)
• allow list creation of decimals (#13851)
• ensure kwargs filter behaviour matches docstring (expect equivalence with eq) (#13864)
• Implement abs for Decimal, error on Date/Time/Datetime (#13821)
• rolling nested groups deadlock (#13835)
• gather_every should work on agg context (#13810)
• Fix segfault of is_in (#13814)
• don't panic on full null qcut (#13815)
• validate operator arithmetic with None, fix Series edge-case (#13780)

📖 Documentation

• Add missing doc entries (#14006)
• add missing len to rst file (#13999)
• Improve structure of user guide (#13951)
• Improve structure of user guide (#13639)
• Introduce ecosystem page in user guide (#13903)
• Mention deltalake write support in README (#13890)
• use proper argument names in the code blocks of api.rst (#13866)

🛠️ Other improvements

• make Enums an actual datatype (#14011)
• omit implicit 'site' from import-timing test (#14009)
• Constructor improvements - part 1 (#14001)
• Add glimpse test (#13979)
• Move PyO3 ChunkedArray conversion logic into its own module (#13973)
• Fix xdist streaming group (#13974)
• Fix spurious test failures (#13961)
• minor describe tidy-up, and slight rewording of some Exception docstrings (#13942)
• Fix pip warning filter return code (#13935)
• Minor refactor of PyO3 conversions module (#13929)
• move filter to polars-compute (#13897)
• Revert pandas warning filter (#13893)
• Make functions in expr/general non-anonymous (#13832)
• Fix doctests (#13831)
• Refactor Python release workflow (#13807)

Thank you to all our contributors for making this release possible!
@ByteNybbler, @JulianCologne, @MarcoGorelli, @Wainberg, @alexander-beedie, @c-peters, @dependabot, @dependabot[bot], @edavisau, @flisky, @ion-elgreco, @itamarst, @jacksonthall22, @kstoneriv3, @mcrumiller, @mkucijan, @nameexhaustion, @orlp, @petrosbar, @r-brink, @reswqa, @ritchie46, @stinodego, @taki-mekhalfa, @thomasaarholt and @valorien

Python Polars 0.20.6-rc.1

🏆 Highlights

• new implementation for String/Binary type. (#13748)

🚀 Performance improvements

• speedup boolean filter (#13905)
• speedup binview filter (#13902)
• allow python threads in read_ functions (#13886)
• improve binview filter (#13878)
• apply string view GC more conservatively (#13850)
• add optimized BinaryViewArray comparison kernels (#13839)
• lazy cache binview bytes len (#13830)
• fast-path for eager int_range (#13811)
• Optimize arr.sum for inner non-null bool (#13800)

✨ Enhancements

• register 'set_sorted' as batch/elementwise (#13896)
• move Enum/Categorical categories to binview (#13882)
• Add ignore_nulls for list.join (#13701)
• Add ignore_nulls for pl.concat_str (#13877)
• Align int_range and int_ranges signatures (#13867)
• fix parquet for binview (#13873)
• support mmap for binview in OOC (#13872)
• implement ffi for binview (#13871)
• Support zero fill null strategy for binary and string columns (#13869)
• allow df.rename and lf.rename to take a renaming function (#13708)
• Implement/fix unary minus operator -pl.col(...) (#13776)
• extend SQL EXTRACT with "century", "millennium", and "timezone" parts (#13634)
• fix binview ipc format (#13842)
• add SQL support for numeric and/or decimal types (#13739)
• improve panic message (#13836)
• Expressify str.zfill (#13790)
• new implementation for String/Binary type. (#13748)
• Add typing to hvplot plot namespace (#13813)
• Add nulls_last for Series.sort (#13794)
• allow ftp URLs, improve URL check (#13781)

🐞 Bug fixes

• prune emtpy chunks before set operations (#13898)
• treat null columns as zero in sum_horizontal (#13880)
• include null count in rolling window validity with min_periods (#13863)
• Fix interchange protocol for new String type (#13881)
• parquet hybrid RLE encoding did not always align to bit width (#13883)
• Add ignore_nulls for list.join (#13701)
• .dt.time() was panicking for datetimes prior to unix epoch (#13812)
• allow list creation of decimals (#13851)
• ensure kwargs filter behaviour matches docstring (expect equivalence with eq) (#13864)
• Implement abs for Decimal, error on Date/Time/Datetime (#13821)
• rolling nested groups deadlock (#13835)
• gather_every should work on agg context (#13810)
• Fix segfault of is_in (#13814)
• don't panic on full null qcut (#13815)
• validate operator arithmetic with None, fix Series edge-case (#13780)

📖 Documentation

• Mention deltalake write support in README (#13890)
• use proper argument names in the code blocks of api.rst (#13866)

🛠️ Other improvements

• move filter to polars-compute (#13897)
• Revert pandas warning filter (#13893)
• Make functions in expr/general non-anonymous (#13832)
• Fix doctests (#13831)
• Refactor Python release workflow (#13807)

Thank you to all our contributors for making this release possible!
@ByteNybbler, @MarcoGorelli, @Wainberg, @alexander-beedie, @dependabot, @dependabot[bot], @edavisau, @flisky, @ion-elgreco, @itamarst, @kstoneriv3, @mcrumiller, @mkucijan, @nameexhaustion, @orlp, @reswqa, @ritchie46, @stinodego, @taki-mekhalfa and @thomasaarholt

Python Polars 0.20.5

⚠️ Deprecations

• Deprecate default delimiter value for str.concat (#13690)
• Rename pl.count() to pl.len() (#13719)
• Deprecate dt.with_time_unit in favor of cast(pl.Int64).cast(pl.Datetime(time_unit, time_zone)) (#13667)

🚀 Performance improvements

• directly embed data ptr in Buffer (#13744)

✨ Enhancements

• Impl count_matches for array namespace (#13675)
• Add nulls_last for list/array.sort (#13795)
• convert fixed-offset timezones to respective Etc timezone from time zone database (#13738)
• allow read_excel to load from remote http locations (#13753)
• Expressify str.slice (#13747)
• implement binview for polars-row (#13736)
• implement binview for polars-json (#13737)
• add architecture for polars-flavored IPC (#13734)
• implement binview comparison kernels (#13715)
• raise default frame/series repr height from 8 to 10 (#13699)

🐞 Bug fixes

• do not read data for zero-length compressed buffer (#13791)
• Fix the non-null test of transpose (#13783)
• Raise error instead of panic when joining on wildcard/nth (#13742)
• str.concat correctly ignore single null value (#13751)
• Selectors by_name and by_dtype should allow empty list as input (#11024)
• Keep Series attributes docstrings when read by Sphinx (#13731)
• fix error message when creating DataFrame from 0-dimensional NumPy array (#13729)
• support corr() for single-column DataFrames (#13728)
• Use NonZeroUsize for batch_size parameter in write_csv/sink_csv/scan_ndjson (#13726)
• error instead of panicking in sql if empty function (#13691)

📖 Documentation

• Fix typo in deprecation message of with_row_count (#13793)
• Fix incorrect "coming from pandas" syntax (#13767)
• Improve streaming section of the user guide (#13750)
• improve n_unique and approx_n_unique docs (#13752)
• add missing Series.str.find reference (#13717)
• Be more explicit about behaviour in str.strip_chars / strip_chars_start / strip_chars_end docstrings (#13697)
• Add doc example for datetime_ranges (#13695)
• document %A and %B to get day name and month name (#13678)

🛠️ Other improvements

• Make pl.duration non-anonymous (#13762)
• Add test for describe on Object types (#13689)
• Only run bytecode parser CI workflow for Python 3.9/3.10 (#13664)

Thank you to all our contributors for making this release possible!
@29antonioac, @MarcoGorelli, @NedJWestern, @Wainberg, @alexander-beedie, @cgevans, @henryharbeck, @langestefan, @orlp, @petrosbar, @r-brink, @reswqa, @ritchie46, @stinodego and @universalmind303

Python Polars 0.20.4

⚠️ Deprecations

• Fix group keys in partition_by(as_dict=True) / GroupBy.__iter__ in some cases (#13646)
• Rename row_count_name/row_count_offset parameters in IO functions to row_index_* (#13563)
• Deprecate dt.datetime in favor of dt.replace_time_zone(None) (#13520)
• Rename with_row_count to with_row_index (#13494)
• Deprecate Expr.where in favor of filter (#13440)
• Allow drop with no inputs as a no-op (#13460)

🚀 Performance improvements

• elide parallelism restriction on generic rolling expressions (#13662)
• ensure time groups are parallelized (#13660)
• do not eagerly compute bitcount (#13562)
• optimise SQL engine string concat (#13499)
• Refactor expression parsing logic of predicates/constraints (#13468)
• Represent Enum categories as Series (#13434)
• remove lifetime requirement from CategoricalChunkedBuilder (#13319)

✨ Enhancements

• write parquet ColumnOrder (#13672)
• Impl contains for ArrayNameSpace (#13638)
• improve rolling() expression formatting (#13657)
• Implement is_between in Rust (#11945)
• Add base PolarsError and PolarsWarning class (#13615)
• typing overloads for Series operator methods ge, gt, ... (#13167)
• Expressify pattern of str.extract (#13607)
• Impl join for ArrayNameSpace (#13586)
• add SQL engine support for string cast to json (#13624)
• add SQL engine support for EXTRACT and DATE_PART (#13603)
• Allow drop with no inputs as a no-op (#13460)
• add SQL engine support for POSITION and STRPOS (#13585)
• additional multi-column support for pl.<function> entries (#13336)
• is_in support for array dtype (#13559)
• add new str.find expression, returning the index of a regex pattern or literal substring (#13561)
• Impl and dispatch arr.first/last to get (#13536)
• Implement from_dataframe natively (interchange protocol) (#10701)
• add SQL engine support for LIKE and ILIKE pattern matching (#13522)
• improve hive partition pruning (#13358) (#13426)
• Add compact syntax for int_range starting from 0 (#13530)
• don't rechunk by default in lazy scans (#13518)
• Add cum_count expression function (#13478)
• add SQL engine support for IF control flow function (#13491)
• add SQL engine support for MOD function (#13502)
• return datetime for datetime mean & median (#13417)
• add SQL engine support for CONCAT_WS string function (#13483)
• Allow map_batches to auto-convert output NumPy arrays to Series (#13277)
• add SQL engine support for RIGHT and REVERSE string functions (#13461)
• implement BinaryView and Utf8View in polars-arrow (#13243)
• add SQL engine support for variadic string CONCAT function (#13428)
• add support for AND in SQL join-clause context (#13242)
• Impl ordering ops for array namespace (#13414)
• add SQL engine support for REPLACE string function (#13431)
• add SQL engine support for SIGN function (#13429)
• add SQL engine support for IFNULL function (#13432)
• additional SQL support for bytes, bit, and hex literals (#13389)

🐞 Bug fixes

• gather.get schema (#13679)
• Fix group keys in partition_by(as_dict=True) / GroupBy.__iter__ in some cases (#13646)
• ensure we hit proper cache in nested rolling expressions (#13666)
• Allow av_buffer cast numeric record to temporal type (#13661)
• streaming cross join if swapped is hit (#13656)
• Make sure rolling key is projected when process projection (#13622)
• fix schema inference for json (#13637)
• Improve parsing of inputs for Expr dunders (#13635)
• Empty series of AggregatedList should also have list dtype (#13620)
• Series.eq_missing should return an Expr when the input is an Expr (#13628)
• fallback to cast kernel if inline_cast AnyValue raise (#13595)
• Fix formatting in describe for precise quantiles (#13593)
• fix reverse variable row decoding (#13587)
• Fix scatter for null values (#13578)
• Fix cum_count with regards to start value / null values (#13535)
• Fix precision/scale handling and invalid numbers in string-to-decimal conversions. (#13548)
• Treat Python None as null value for Object dtype (#13564)
• Fix scatter to allow single temporal inputs (#13577)
• Fix interchange protocol data buffer dtype (#10787)
• Expr.replace to single value did not replace NULLs (#13551)
• improve hive partition pruning (#13358) (#13426)
• fix projection pushdown for new outer join schema (#13527)
• dont raise when partial function is passed to map_elements (#13524)
• improve reading of mixed string/other dtype column data from spreadsheets with openpyxl and pyxlsb engines (#13495)
• ensure size-hint of TrueIdxIter is correct (#13508)
• correct 'outer_coalesce' logic in case of duplicate names (#13501)
• raise for out-of-range datetimes in to_datetime/strptime (#13403)
• Fix Series equality for List/Array types (#13477)
• Keep logical type when getting values from list (#13456)
• Handle duplicate/ambiguous inputs for replace (#13217)
• Handle empty inputs to Enum constructor (#13446)
• Fix group_by iteration when grouping by certain selectors (#13437)
• Fix to_pandas for 0x0 dataframe (#13420)
• Fix offsets for numeric types in from_buffer (#13398)

📖 Documentation

• Clarify documentation for the agg_list argument in Expr.map_batches (#13625)
• fix linking to feature flags in user guide (#13644)
• bring sink_ndjson docstring in line with other sink docstrings (#13636)
• Update then and otherwise docstrings with "strings are parsed as column names" (#13630)
• Add sink_ndjson to API reference. (#13627)
• Improve documentation on broadcasting (#13394)
• Add note about toolchain issue under native Windows (#13590)
• Hint about ruff setting in VSCode (#13421)
• Clarify examples for .transpose() (#13581)
• Add additional Series docstring examples (#13558)
• Doc example for read_csv (#13161) (#13545)
• Add more doc examples on how to create an index column (#13532)
• update SQL section of the README (#13529)
• Add note to int_range docs for creating an index column (#13516)
• add a note to the read_database_uri docstring about escaping special characters in the connection string (#13514)
• update polars-business > polars-xdt link (#13509)
• Fix various typos, grammar and formatting in docstrings and user guide (#13506)
• Doc examples for threadpool_size and get_index_type (#13496)
• Add missing datetime examples to docs (#13487)
• add polars-distance to plugins page (#13454)
• define file-like object in read_parquet docstring (#13463)
• Move Series.struct.json_encode to methods in Sphinx autosummary (#13443)
• Add missing examples of series/list.py (#13423)
• show datetime.date import in code block (#13419)
• clarify documentation for rle and rle_id (#13397)
• use named series in Series.plot example (#13407)
• fix alphabetical order of documentation entries (#13396)

🛠️ Other improvements

• Auto-add 'needs triage' label to bugs (#13671)
• make rolling index column visible to optimizer (#13658)
• Enable new error message lint to improve stack trace display (#13596)
• Add Documentation / Build system sections to the changelog (#13594)
• Filter unhelpful messages in make build (#13579)
• Remove extra line break between checkboxes in GitHub bug report issues (#13576)
• Narrow type hint for get_index_type util (#13556)
• Fix some test failures/slowdowns (#13504)
• pandas 2.2 compat (#13467)
• Increase timeout for gevent async test (#13448)
• Do not end docstrings with a blank line (#13193)

Thank you to all our contributors for making this release possible!
@Bromeon, @MarcNuebel, @MarcoGorelli, @ShivMunagala, @Wainberg, @aaarrti, @alexander-beedie, @bchalk101, @c-peters, @cgevans, @cmdlineluser, @collinprince, @deanm0000, @hamishs, @henryharbeck, @ion-elgreco, @jcrozum, @mcrumiller, @nameexhaustion, @orlp, @petrosbar, @r-brink, @reswqa, @ritchie46, @s-banach, @shritesh, @stinodego, @tim-stephenson and @wjandrea

Rust Polars 0.36.2

🏆 Highlights

• Add new Enum categorical data type which allows a fixed set of categories (#11822)

💥 Breaking changes

• Rename Utf8 data type to String (#13224)
• Rename set_at_idx to scatter (#12687)
• Preserve left and right join keys in outer joins (#12963)
• Implement dtype parameter for int_range on Rust side (#12940)
• Update Expr.count to ignore null values by default (#12934)
• Change value_counts resulting column name from counts to count (#12506)
• Change default join behavior with regard to nulls, add join_nulls parameter to keep existing behavior (#12840)
• Smaller integer data types for datetime components (#12070)
• Fix NaN ordering to make NaNs compare greater than any other float, and equal to themselves (#12721)
• Rename frame_equal/series_equal to equals (#12663)
• Rename not_ expression to not on the Rust side (#12587)
• Rename str.json_extract to str.json_decode (#12586)
• Rename DataFrame column index methods (#12542)

🚀 Performance improvements

• optimize set bit count (#13317)
• speed up .dt.truncate for large numbers of years (#13310)
• don't eagerly evaluate error branches (#13311)
• don't needlessly allocate validity in concat/rechunk (#13288)
• add fast path to count_bits_set_by_offsets (#13253)
• make .dt.truncate('*mo') more than 3x faster (#13192)
• ensure single expression evaluation for replace (#13147)
• Elide allocation in outer join materialization (#12992)
• Ensure we reduce for any/all_horizontal (#12976)
• Add fast paths for UTC in truncate (#12965)
• Improve rolling_median algorithm (#12704)
• Use fast path for non-null data in new SQL-like null matching (#12874)
• improve merge_local_rhs_categorical traversal (#12660)
• make values_size estimate correct for sliced arrays (#12658)
• improve parquet utf8 validation (#12655)
• parquet pre-allocate buffer in binary plain encode (#12652)
• optimize dict binary decoding in parquet (#12648)
• ensure we only check the values within bounds (#12633)
• parquet; elide recursion in hot path (#12625)
• improve cov/corr algorithm (#12590)
• apply left side predicate pushdown also to right side on semi join (#12565)
• ensure streaming parquet download remains concurrent ~7x (#12552)
• speed up parquet download of streaming engine (#12544)

✨ Enhancements

• support negative indices in gather in group_by context (#13373)
• support negative indexing in gather (select context) (#13343)
• support min_periods for temporal rolling aggregations (#13342)
• support REGEXP and RLIKE pattern matching in SQL engine (#13359)
• gracefully handle panics in plugins (#13329)
• Implement unique/n_unique/unique_counts/is_unique/is_duplicated for Null series (#13307)
• support common variant spelling STDEV in the SQL engine (in addition to STDDEV) (#13303)
• change doc links to new url docs.pola.rs (#13290)
• support horizontal concatenation of LazyFrames (#13139)
• Impl serde for array dtype (#13168)
• dispatch strict_cast via cast (#13255)
• Impl any/all for array type (#13250)
• add cancellable queries (#13178)
• add offset parameter to gather_every (#13156)
• Support Array dtype AnyValue Series construction (#12817)
• Allow step parameter in int_ranges to take an expression (#13148)
• Implement count for DataFrame/LazyFrame (#13153)
• Move from GA to more privacy friendly framework (#13155)
• Rename set_at_idx to scatter (#12687)
• prune all/any_horizontals with single inputs (#13146)
• ensure we get cleaner logical plans with any/all_horizontal (#13144)
• Add str.contains_any and str.replace_many (Aho-Corasick algorithms) (#13073)
• Auto-infer credentials from .aws folder (#13062)
• Support private cloud S3 storage in scan_parquet (#13060)
• Allow order operators (<,>,>=,<=) on Enum types (#12982)
• Reimplement replace expression on the Rust side (#13002)
• Use tokio semaphore for concurrency handling (#13026)
• Improve and expressify hist (#13014)
• Preserve left and right join keys in outer joins (#12963)
• Allow end before start in date/time_range (#12964)
• Implement group-tuples for Null dtype (#12975)
• Implement dtype parameter for int_range on Rust side (#12940)
• Cast to an enum from int (#12954)
• Move categorical ordering into dtype (#12911)
• Update Expr.count to ignore null values by default (#12934)
• Enable partial predicate pushdown past window expressions (#12710)
• Add str.reverse (#12878)
• Change value_counts resulting column name from counts to count (#12506)
• Implement std and var for Duration columns (#12865)
• Change default join behavior with regard to nulls, add join_nulls parameter to keep existing behavior (#12840)
• Preserve base dtype when raising to UInt power (#10446)
• Smaller integer data types for datetime components (#12070)
• Support SQL subqueries for JOIN and FROM (#12819)
• parquet support required deltabyte encoding (#12836)
• Add new Enum categorical data type which allows a fixed set of categories (#11822)
• support nested null in vstack/append/extend/concat (#12771)
• Improve error messages on attempted Arrow conversions involving incompatible/unknown dtypes (#12421)
• determine mode parallelism depending on current tasks (#12764)
• enable slice push down past with_columns (#12742)
• implement From<LazyGroupBy> for LazyFrame (#12562)
• Rename frame_equal/series_equal to equals (#12663)
• Join operations on local categoricals (#12657)
• use RLE_DICTIONARY for integers in parquet (#12647)
• Add configuration option for where Polars spills to disk (#12595)
• implement RLE_DICT encoding for utf8/binary columns (reduced parquet file size) (#12623)
• implement 'DeltaByteArray' decoding for parquet (#12602)
• warn if by column is not sorted in rolling aggregations (as opposed to raising), add warn_if_unsorted argument (#12398)
• struct -> json encoding expression (#12583)
• Implement support for multi-character comments in read_csv (#12519)
• Implement LazyFrame.sink_ndjson (#10786)
• improve concurrency parameters (#12567)
• Adds sink_ipc_cloud (#12556)
• Adds sink_ipc_cloud (#11008)
• In explain(), rename PIPELINE to STREAMING so it's clearer what it means (#12547)

🐞 Bug fixes

• range/ranges output name should follow lhs rule (#13369)
• updated Display trait for enum categoricals (#13331)
• nested dtypes: export logical type in plugins (#13325)
• fix invalid dtype setting in array (#13327)
• fix csv parser error when commented-out rows precede the header row (#13318)
• invalid schema outer join after projection pd (#13315)
• invalid predicate optimization (#13313)
• Account for null values in categorical unique/n_unique (#13308)
• fix schema when subtracting (#13309)
• broadcasting of unit LHS in string operations (#12737)
• casting list/arr to arr/list shouldn't convert chunks to logical type (#13259)
• sorting categorical lexically bugs on null values (#13271)
• improve replace on categoricals (#13223)
• round trip to JSON and back should preserve Enum type (#13267)
• enable and fix SIMD in polars-compute (#13251)
• match_chunks shouldn't change the dtype (#13222)
• sink_csv deadlock (#13239)
• is_in operator for categoricals (#13205)
• Better handle mismatched dtypes in replace (#13213)
• Fix replace fast path by casting old input to the right data type (#13176)
• ndjson nested null schema inference (#13206)
• slice for NullChunked no longer force single chunk (#13174)
• don't cast to unknown dtypes (#13197)
• Allow casting nullable list to array (#13196)
• maintain old join behavior in window expression (#13179)
• Fix comparison of categoricals (#13137)
• Use the name of the leftmost expression in horizontal operations (#13143)
• any_value should supports cast to boolean (#13125)
• Update offsets of null value correctly for all from_iter_xxx_trusted_len (#13132)
• fix neq for series cmp str (#13128)
• fix category list builder append series with multiple chunks (#13116)
• repeat_by should not raise if by contains nulls (#13105)
• [csv] raise on single quote char (#13104)
• Raise if scan zstd compressed csv file (#13102)
• Don't check map length if input is literal (#13098)
• use FunctionExpr's scalar return type for is_in (#13091)
• rolling_quantile can get incorrect state (#13088)
• Fix off-by-one error in quantile(method="nearest") (#13058)
• Fix incorrect schema inference on nested columns (#13057)
• Don't raise for datetime_range if starting on ambiguous datetime and earliest was specified (#13050)
• add cast safety to literals (#12983)
• Parse json_decode per max buffer length (#13029)
• Parse 00:00 time zone as UTC (#13034)
• Fix timeout errors in concurrent downloads (#13023)
• Fix SQL substring indexing (#13016)
• Allow broadcasting in ranges (#11900)
• Prevent deadlock in sink_csv (#12991)
• Don't get mutable if buffer is sliced (#12979)
• Dataframes with Decimal columns cannot be pickled (#12955)
• Fix truncate when truncating by multiple weeks (#12948)
• Fix segfault / memory corruption after plugins return Err result (#12953)
• Don't panic when ambiguous parameter is not Utf8 (#12913)
• don't panic on empty df in merge_sort (#12923)
• Patch rolling_var/rolling_std numerical stability (#12909)
• Fix incorrect Int16 min/max due to incorrect SIMD mask construction (#12908)
• Fix OOB error in list set operations on empty frame (#12845)
• Fix repr of Expr.gather (which was still showing deprecated take) (#12864)
• Fix nan_min/max incorrectly aggregating chunks with addition (#12848)
• write only one dict page per row rowgroup (#12831)
• incorrect values from parquet RLE decoding (#12818)
• Handle aggregation for all-NaN groups in group_by (#12304)
• Use total float ordering ...

Python Polars 0.20.3

🏆 Highlights

• add plot namespace (which defers to hvplot) (#13238)

🚀 Performance improvements

• optimize set bit count (#13317)
• speed up .dt.truncate for large numbers of years (#13310)
• don't eagerly evaluate error branches (#13311)
• don't trigger internal borrwing in numpy memmap (#13304)
• don't needlessly allocate validity in concat/rechunk (#13288)
• add fast path to count_bits_set_by_offsets (#13253)
• make .dt.truncate('*mo') more than 3x faster (#13192)

✨ Enhancements

• support negative indices in gather in group_by context (#13373)
• validate Enum categories (#13356)
• improve Series/DataFrame init from existing Series/DataFrame objects (#13344)
• support negative indexing in gather (select context) (#13343)
• support min_periods for temporal rolling aggregations (#13342)
• support REGEXP and RLIKE pattern matching in SQL engine (#13359)
• emit suggestion for how to replace map_elements sigmoid function with expressions (#13347)
• Support Enum types in interchange protocol (#13368)
• add plot namespace (which defers to hvplot) (#13238)
• gracefully handle panics in plugins (#13329)
• rework pl.exclude as a pure selector, allowing other selectors as input (#13301)
• Implement unique/n_unique/unique_counts/is_unique/is_duplicated for Null series (#13307)
• support common variant spelling STDEV in the SQL engine (in addition to STDDEV) (#13303)
• enhance expression-level filter syntax with support for multiple predicates and kwargs (#12689)
• change doc links to new url docs.pola.rs (#13290)
• support horizontal concatenation of LazyFrames (#13139)
• Rename Utf8 data type to String, keep Utf8 as alias (#13257)
• dispatch strict_cast via cast (#13255)
• Impl any/all for array type (#13250)
• add cancellable queries (#13178)
• add offset parameter to gather_every (#13156)
• Support Array dtype AnyValue Series construction (#12817)
• Allow step parameter in int_ranges to take an expression (#13148)
• make python map_batches safer (#13181)
• Implement count for DataFrame/LazyFrame (#13153)

🐞 Bug fixes

• don't lose track of ones and zeros dtype, improve use with Array, raise error if dtype invalid (#13326)
• updated Display trait for enum categoricals (#13331)
• nested dtypes: export logical type in plugins (#13325)
• fix invalid dtype setting in array (#13327)
• fix csv parser error when commented-out rows precede the header row (#13318)
• invalid schema outer join after projection pd (#13315)
• invalid predicate optimization (#13313)
• Account for null values in categorical unique/n_unique (#13308)
• fix schema when subtracting (#13309)
• broadcasting of unit LHS in string operations (#12737)
• sorting categorical lexically bugs on null values (#13271)
• improve replace on categoricals (#13223)
• round trip to JSON and back should preserve Enum type (#13267)
• fix return type hint of list series any/all (#13265)
• sink_csv deadlock (#13239)
• Correctly use read_parquet for all binary inputs (#13218)
• is_in operator for categoricals (#13205)
• Better handle mismatched dtypes in replace (#13213)
• Fix replace fast path by casting old input to the right data type (#13176)
• ndjson nested null schema inference (#13206)
• don't cast to unknown dtypes (#13197)
• maintain old join behavior in window expression (#13179)

🛠️ Other improvements

• reverse condtion order in udfs _expr function (#13348)
• Update release workflow for new upload/download artifact versions (#13355)
• Allow construction of Series from memory buffers (#13323)
• add 'pipe littering' to 'coming from pandas' section (#13335)
• Refactor functionality related to Series buffers (#13291)
• Restore light/darkmode switch in API reference (#13312)
• Copy Makefile build commands to top level (#13293)
• Fix release flags (#13298)
• Re-enable consortium standard tests (#13296)
• Update CODEOWNERS (#13292)
• Add CPU compatibility check (#13134)
• Change base url of docs/guide to docs.pola.rs (#13281)
• Fix source link for dev docs (#13279)
• fix return type hint of list series any/all (#13265)
• Fix display of overloaded signatures (#13258)
• clean up bytecode parsing a bit (#13221)
• Add a couple of docstring examples to Series methods (#13244)
• remove unnecessary arg unpacking (#13241)
• update rustc (#13219)
• fix horizontal concatenation documentation (#13141)
• Replace blackdoc by ruff's new docstring formatter (#13182)
• Update ruff & ruff settings (#13126)
• Link to latest object_store docs in api doc (#13180)
• Fix failing test (#13171)

Thank you to all our contributors for making this release possible!
@MarcoGorelli, @TNieuwdorp, @adamreeve, @alexander-beedie, @c-peters, @cjfuller, @dependabot, @dependabot[bot], @mcrumiller, @nameexhaustion, @orlp, @petrosbar, @r-brink, @reswqa, @ritchie46, @robvanmieghem and @stinodego