2 Releases published by 1 person

• rs-0.42.0 Rust Polars 0.42.0

  published Aug 14, 2024

• py-1.5.0 Python Polars 1.5.0

  published Aug 14, 2024

46 Pull requests merged by 16 people

• feat: Allow float in interpolate_by by column

  #18015 merged Aug 18, 2024

• refactor(rust): Add multiplexer node to new streaming engine

  #18241 merged Aug 16, 2024

• refactor(rust): Add feature gates for polars-python crate

  #18232 merged Aug 16, 2024

• refactor: Split py-polars crate

  #18204 merged Aug 16, 2024

• test(python): Fix test for new deltalake release

  #18211 merged Aug 16, 2024

• chore: Update the required version of rust in README.md

  #18203 merged Aug 16, 2024

• docs(python): Fix column name output in example of DataFrame.map_rows

  #18227 merged Aug 16, 2024

• fix(python): Convert date and datetime in literal construction

  #16018 merged Aug 15, 2024

• test(python): Fix version bifurcation for test_read_database_cx_credentials

  #18220 merged Aug 15, 2024

• refactor(rust): Add itertools in utils

  #18213 merged Aug 15, 2024

• feat(python): Make show_versions more responsive

  #18208 merged Aug 15, 2024

• perf: Speedup ndjson reader ~40%

  #18197 merged Aug 15, 2024

• fix(python): Fix gather str as lit

  #18207 merged Aug 15, 2024

• chore: Use or_else for raising

  #18206 merged Aug 15, 2024

• chore: Remove unused Parquet source files

  #18193 merged Aug 14, 2024

• Rust Polars 0.42.0

  #18195 merged Aug 14, 2024

• perf: Skip parquet page when unneeded

  #18192 merged Aug 14, 2024

• refactor(rust): Consistently use PlRandomState, PlHashMap/Set

  #18190 merged Aug 14, 2024

• fix: Fix struct shift and list builder

  #18189 merged Aug 14, 2024

• fix: Don't load Parquet nested metadata

  #18183 merged Aug 14, 2024

• fix: Throw bigidx error for Parquet row-count

  #18154 merged Aug 14, 2024

• feat: Create literals for datetime/date expressions

  #18184 merged Aug 14, 2024

• fix: Fix unpivot on empty df

  #18179 merged Aug 14, 2024

• feat: Create literals in 'datetime' expression

  #18182 merged Aug 14, 2024

• feat(rust): Add missing impl for Series

  #18166 merged Aug 14, 2024

• fix: Don't vertically parallelize cse contexts

  #18177 merged Aug 14, 2024

• docs(python): Fix references to old methods in lazy docstring

  #18178 merged Aug 14, 2024

• fix(python): Ensure default values are included when saving/restoring the current Config state

  #18151 merged Aug 14, 2024

• chore: Remove unneeded growable

  #18165 merged Aug 14, 2024

• docs(python): Include PyCapsule Interface in DataFrame and Series API docs

  #18174 merged Aug 14, 2024

• feat(python): Expose top-level "has_header" param for read_excel and read_ods

  #18078 merged Aug 13, 2024

• docs(python): Corrected example result in group_by docs

  #18169 merged Aug 13, 2024

• perf: Improve binview extend/ifthenelse

  #18164 merged Aug 13, 2024

• perf: Start on better Parquet delta decoding

  #18049 merged Aug 13, 2024

• refactor(rust): Allow non-scoped tasks to be spawned

  #18163 merged Aug 13, 2024

• fix: Properly handle empty Parquet row groups with no dictionary

  #18161 merged Aug 13, 2024

• perf(python): Rechunk group-by __iter__

  #18162 merged Aug 13, 2024

• fix: Struct outer nullabillity

  #18156 merged Aug 13, 2024

• chore: Update Cargo.lock to fix build error on Linux

  #18153 merged Aug 13, 2024

• perf: Tune jemalloc to not create muzzy pages

  #18148 merged Aug 12, 2024

• feat: Raise on invalid 'is_between' and improve error message quality

  #18147 merged Aug 12, 2024

• fix: Fix pyarrow predicate pushdown regression

  #18145 merged Aug 12, 2024

• perf: Reduce default async thread count

  #18142 merged Aug 12, 2024

• fix: Prevent unwanted supertype cast in 'search_sorted'

  #18143 merged Aug 12, 2024

• fix: Parquet with filter=None

  #18139 merged Aug 12, 2024

• fix: Don't raise when converting from pandas if index contains duplicate names when include_index=False (the default)

  #18133 merged Aug 11, 2024

17 Pull requests opened by 11 people

• refactor: Allow Python UDFs in Polars Cloud plans

  #18135 opened Aug 11, 2024

• refactor(rust): Add parquet source node to new streaming engine

  #18152 opened Aug 13, 2024

• docs: Possibly fix table display issue

  #18187 opened Aug 14, 2024

• feat: Allow whitespace with `to_integer`

  #18196 opened Aug 14, 2024

• fix(python): Handle boolean comparisons in Iceberg predicate pushdown

  #18199 opened Aug 15, 2024

• fix: Fix panic streaming parquet scan from cloud with slice

  #18202 opened Aug 15, 2024

• fix: Raise suitable error on unsupported `SQL` set op syntax

  #18205 opened Aug 15, 2024

• feat(python): Allow mapping as syntactic sugar in `str.replace_many`

  #18214 opened Aug 15, 2024

• feat(python): Support "indexed" dicts as input to `from_dict` and `from_dicts`

  #18215 opened Aug 15, 2024

• fix: Categorical `unpivot`

  #18216 opened Aug 15, 2024

• docs(python): Add missing "Parameters" section to `bin.size` expr docstring

  #18222 opened Aug 15, 2024

• perf: Use direct path for `time`/`timedelta` literals

  #18223 opened Aug 15, 2024

• docs(python): Improve formatting and consistency of various docstrings

  #18237 opened Aug 16, 2024

• fix(rust): Make json readers ignore BOM character

  #18240 opened Aug 16, 2024

• fix: Consistently round down in dt.round

  #18245 opened Aug 17, 2024

• fix: Compute joint null mask before calling rolling corr/cov stats

  #18246 opened Aug 18, 2024

• feat: Support empty structs

  #18249 opened Aug 18, 2024

44 Issues closed by 12 people

• Interpolate based on other Float64 column

  #16794 closed Aug 18, 2024

• `pivot` docs do not describe different between `index` and `values` arguments

  #17967 closed Aug 17, 2024

• Expr `.implode().get().over()` has strange results.

  #18228 closed Aug 16, 2024

• [Python] Scan_pyarrow_dataset() no longer pushes down partition filtering to pyarrow

  #14343 closed Aug 16, 2024

• Upgrading to 0.42 breaks compilation

  #18226 closed Aug 16, 2024

• Cannot load CSV file with single column and empty lines at start

  #18071 closed Aug 16, 2024

• `dtype` ingored in `pl.lit` between date and datetime

  #16012 closed Aug 15, 2024

• `test_read_database_cx_credentials` expected exception does not bifurcate over correct Python version

  #18218 closed Aug 15, 2024

• release new Rust Polars version on crates.io?

  #18125 closed Aug 15, 2024

• Request to return LazyFrame for pl.from_arrow and more

  #18212 closed Aug 15, 2024

• There are still methods with str_as_lit=True thereby treating strings as literal instead of column, is this intentional?

  #18099 closed Aug 15, 2024

• Polars does not allow reading a dataframe from piped csv output of another process

  #17927 closed Aug 15, 2024

• DataFrame construction from numpy with dtype object

  #17819 closed Aug 15, 2024

• `.over()` fails with `.get`.

  #18191 closed Aug 14, 2024

• `df.with_columns(pl.col('struct_col').shift(-1).over('group_col')` raises an error

  #18107 closed Aug 14, 2024

• Wrong result when filtering categorical using `.is_in` in `scan_parquet`

  #18188 closed Aug 14, 2024

• Incorrect statistics are loaded from parquet containing structs

  #18136 closed Aug 14, 2024

• Throw Exception Reading > UInt32 Row Datasets

  #18149 closed Aug 14, 2024

• SQL expression panics on invalid query

  #18172 closed Aug 14, 2024

• When using `.implode().list` in `GroupBy.agg`, the return type is list, not the original type of the value.

  #18176 closed Aug 14, 2024

• `unpivot()` on an empty DataFrame creates two empty `variable` columns

  #18170 closed Aug 14, 2024

• `Series.search_sorted` gives wrong answer when using expression arguments

  #18180 closed Aug 14, 2024

• Getting a "ComputeError: cannot evaluate two Series of different lengths" with straight forward Lazy expressions.

  #18124 closed Aug 14, 2024

• Unclear whether one should use `.describe(_optimized)_plan()` or `.explain()`

  #18103 closed Aug 14, 2024

• Example code for `group_by` iteration is wrong

  #18168 closed Aug 13, 2024

• "has_header" in "read_options" for `read_excel` has inconsistent behavior across engines

  #16944 closed Aug 13, 2024

• Add `Expr.scatter`

  #18167 closed Aug 13, 2024

• write_database to snowflake with adbc engine spouts context canceled error log

  #17666 closed Aug 13, 2024

• Unable to read empty dataframe from Parquet without dictionary

  #18146 closed Aug 13, 2024

• Nullability of `Struct` series

  #18119 closed Aug 13, 2024

• Unexpected results occur when using the head(1) expression in the select method.

  #18155 closed Aug 13, 2024

• Add DRY equivalent for Series

  #18150 closed Aug 13, 2024

• Series is not an instance of Sequence

  #18138 closed Aug 12, 2024

• Slowdown of up to 2x compared to NumPy / Pandas in division operator

  #17414 closed Aug 12, 2024

• polars performance with simple array arithmetic much slower than NumPy?

  #18088 closed Aug 12, 2024

• Do not cast to supertypes for `is_between`

  #18144 closed Aug 12, 2024

• More informative error messages for unsupported options in streaming mode

  #18121 closed Aug 12, 2024

• Predicates with `&` where one side is `is_in(Series)` are no longer pushed down to pyarrow scan

  #18141 closed Aug 12, 2024

• `search_sorted` silently allows comparisons between different types

  #18096 closed Aug 12, 2024

• DataFrame rendering broken on Android

  #18140 closed Aug 12, 2024

• Calling `to_deserializer` with `filter = None` results in a panic

  #18137 closed Aug 12, 2024

• polars.when(<condition>) treats False as True

  #18134 closed Aug 12, 2024

• Pandas DataFrames with a multiindex fails to be converted to a Polars DataFrame

  #18130 closed Aug 11, 2024

• Querying DuckDB using Polars' logical plan

  #17997 closed Aug 11, 2024

34 Issues opened by 29 people

• Polars ignoring rows that are empty in Excel

  #18250 opened Aug 18, 2024

• Inconsistent rolling results when using temporal windows

  #18248 opened Aug 18, 2024

• Dependencies not recognized

  #18247 opened Aug 18, 2024

• read_ndjson ignores provided schema list inner types if values are inferred null

  #18244 opened Aug 17, 2024

• rank() on a Series of just 1 null assigns rank=1 to the null value.

  #18243 opened Aug 16, 2024

• write_parquet with partition_by silently overwrites existing files

  #18242 opened Aug 16, 2024

• `dt.round()` slow/fast path use different rounding

  #18239 opened Aug 16, 2024

• ExprStringNameSpace replace / replace_all literal flag ignored for dataframes with multiple rows

  #18238 opened Aug 16, 2024

• Inconsistent Results Between Pandas and Polars using cut (and qcut)?

  #18236 opened Aug 16, 2024

• Multi-output, multi-sink lazy polars

  #18235 opened Aug 16, 2024

• `.struct.field()` after `shuffle()` seems to produce incorrect results

  #18234 opened Aug 16, 2024

• `Expr.shuffle` uses different order per column

  #18233 opened Aug 16, 2024

• `.first` behind `.sort_by.slice` gets wrong result.

  #18231 opened Aug 16, 2024

• Behaviour change of Expr.list.drop_nulls for structs with all None fields

  #18230 opened Aug 16, 2024

• upsample not working if arguments are only sorted within group

  #18229 opened Aug 16, 2024

• `.implode().list` method is not valid before `.over`.

  #18225 opened Aug 16, 2024

• In `bin.size()`, the `unit` parameter is not described

  #18221 opened Aug 15, 2024

• In `scan_parquet()`, `include_file_paths` returns twice the same column

  #18219 opened Aug 15, 2024

• rolling_corr giving inconsistent results

  #18217 opened Aug 15, 2024

• `compat_level` documentation is unclear

  #18210 opened Aug 15, 2024

• Consistency in check_exact between assert_frame_equal and polars.DataFrame.equals

  #18209 opened Aug 15, 2024

• Examples of using `scan_csv` with cloud URIs

  #18201 opened Aug 15, 2024

• Use of write_ipc with file=None

  #18200 opened Aug 15, 2024

• entered unreachable code when using collect_schema and concat

  #18198 opened Aug 14, 2024

• Don't disable coalesce when joining on expressions

  #18194 opened Aug 14, 2024

• Parquet files cannot be read from pre-signed S3 URLs due to S3 forbidding HTTP HEAD

  #18186 opened Aug 14, 2024

• `Expr.replace` and `Expr.replace_strict` set "NO_DEFAULT" as a value

  #18185 opened Aug 14, 2024

• Panic occurring when using streaming and limit with Parquet

  #18181 opened Aug 14, 2024

• ParquetWriter<CloudWriter> hangs when uploading to S3

  #18175 opened Aug 13, 2024

• Make series raw display output to not use single quotes to be consistent with the DataFrame raw display outputs.

  #18173 opened Aug 13, 2024

• `pl.struct` inside `list.eval` produces different dtype if fields are named

  #18160 opened Aug 13, 2024

• Request for new method argmax_horizontal

  #18159 opened Aug 13, 2024

• Series constructor with NumPy datetime64 scalar returns incorrect results

  #18158 opened Aug 13, 2024

• Add cumulative n_unique

  #18157 opened Aug 13, 2024

44 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• feat(python!): Use Altair in DataFrame.plot

  #17995 commented on Aug 14, 2024 • 6 new comments

• feat: Add `name.replace` to the set of column rename options

  #17942 commented on Aug 14, 2024 • 1 new comment

• Unable to melt categorical-type columns with other types

  #12231 commented on Aug 15, 2024 • 0 new comments

• Panic when converting dataframe to arrow

  #17955 commented on Aug 15, 2024 • 0 new comments

• scan_pyarrow_dataset not filtering on partitions

  #16300 commented on Aug 15, 2024 • 0 new comments

• Split off list constructor logic from `pl.concat_list` into `pl.list`

  #8510 commented on Aug 15, 2024 • 0 new comments

• can write empty parquet but not read

  #13457 commented on Aug 15, 2024 • 0 new comments

• PanicException: implementation error in fill_null

  #18051 commented on Aug 15, 2024 • 0 new comments

• Do not enclose str items in quotes in HTML repr

  #18102 commented on Aug 16, 2024 • 0 new comments

• Ability for str.extract_all to return only capture groups

  #16602 commented on Aug 16, 2024 • 0 new comments

• `fill_null` resulting data type is incorrect in some cases

  #13789 commented on Aug 16, 2024 • 0 new comments

• Normalize JSON

  #7078 commented on Aug 16, 2024 • 0 new comments

• expose `null_on_oob` on `gather`.

  #16842 commented on Aug 17, 2024 • 0 new comments

• `read_json` fails on empty objects

  #10234 commented on Aug 17, 2024 • 0 new comments

• Allow `pl.Expr.get` handle null index values in Aggregation context so that `pl.Expr.arg_min/arg_max` is always valid input

  #15163 commented on Aug 17, 2024 • 0 new comments

• Support for handling duplicate timestamps in rolling operations similar to Pandas

  #18126 commented on Aug 18, 2024 • 0 new comments

• feat(rust, python): Add `cat.to_local()` expression

  #15090 commented on Aug 12, 2024 • 0 new comments

• fix: Use appropriate bins in `hist` when `bin_count` specified

  #16942 commented on Aug 16, 2024 • 0 new comments

• fix(rust): Address replacing all array elements with nulls for `map_elements`

  #17934 commented on Aug 15, 2024 • 0 new comments

• feat: Implement cast from `pl.String` to `pl.Time`

  #18108 commented on Aug 14, 2024 • 0 new comments

• feat: do not check repeated join keys

  #18109 commented on Aug 12, 2024 • 0 new comments

• feat: Add devcontainer for local development and GitHub Codespaces support

  #18132 commented on Aug 17, 2024 • 0 new comments

• Error when joining on the same pair of key names (but from different tables)

  #18007 commented on Aug 11, 2024 • 0 new comments

• Memory leak - Polars with dash/flask

  #18074 commented on Aug 12, 2024 • 0 new comments

• Concatenation of array columns, similar to concat_list

  #18090 commented on Aug 12, 2024 • 0 new comments

• Automatically adjust the number of columns printed for a DataFrame.

  #6105 commented on Aug 12, 2024 • 0 new comments

• Add option for n_unique() to ignore None values

  #17928 commented on Aug 13, 2024 • 0 new comments

• Pandas.Timestamp timezone and unit gets modified when creating new Polars Dataframe

  #18127 commented on Aug 13, 2024 • 0 new comments

• Feature Request: add to `df.write_ndjson(json_columns: list[str])`, for columns to be decoded and written out as JSON

  #17054 commented on Aug 13, 2024 • 0 new comments

• Unimplemented join on list[i64] succeeds with empty dataframe in lazy mode

  #18120 commented on Aug 13, 2024 • 0 new comments

• Provide native and fast Series slice assignment (currently slower than Pandas)

  #17728 commented on Aug 13, 2024 • 0 new comments

• Deadlock when using async parquet reader

  #18086 commented on Aug 13, 2024 • 0 new comments

• Rolling_mean with weights gives wrong result when min_periods is not None

  #17344 commented on Aug 13, 2024 • 0 new comments

• Cache HTTP requests

  #15649 commented on Aug 13, 2024 • 0 new comments

• Error reading ods file with read_ods

  #14053 commented on Aug 14, 2024 • 0 new comments

• Copy logic-plan from one LazyFrame to another LazyFrame?

  #16430 commented on Aug 14, 2024 • 0 new comments

• Allow nested / chained aggregations

  #14361 commented on Aug 14, 2024 • 0 new comments

• Scan Parquet .collect() on Azure Blobs fails when column in different order. ShapeError: unable to vstack, column names don't match

  #13381 commented on Aug 14, 2024 • 0 new comments

• allow `pl.element()` to be used in Series `filter` operation

  #13772 commented on Aug 14, 2024 • 0 new comments

• `df.join_asof()` - give example of what `coalesce` argument does

  #18089 commented on Aug 14, 2024 • 0 new comments

• Transpose option for `DataFrame.describe()`

  #17797 commented on Aug 14, 2024 • 0 new comments

• Make horizontal and vertical versions of and(), any() consistent w.r.t default null strategy

  #12585 commented on Aug 15, 2024 • 0 new comments

• map_elements replaces all array elements with nulls

  #17873 commented on Aug 15, 2024 • 0 new comments

• Polars write database - Rest API call failing with AWS Lambda trigger

  #17880 commented on Aug 15, 2024 • 0 new comments