Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Passing too few names to .struct.rename_fields removes remaining fields. #9052

Open
2 tasks done
cmdlineluser opened this issue May 26, 2023 · 2 comments · May be fixed by #15624
Open
2 tasks done

Passing too few names to .struct.rename_fields removes remaining fields. #9052

cmdlineluser opened this issue May 26, 2023 · 2 comments · May be fixed by #15624
Labels
A-dtype-struct Area: struct data type A-input-parsing Area: parsing input arguments bug Something isn't working P-low Priority: low python Related to Python Polars

Comments

@cmdlineluser
Copy link
Contributor

Polars version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of Polars.

Issue description

I'm not sure if this is a bug or not.

I just noticed that if you supply too few names to struct.rename_fields - the remaining fields are "dropped".

I'm not sure if this is intended behaviour of if this should be a strict operation where the correct amount of names need to be given?

Thanks.

Reproducible example

import polars as pl

df = pl.DataFrame({"A": [[1, 2]], "B": [[3, 4]]})
df = df.with_columns(pl.all().arr.to_struct())

df.with_columns(pl.col("A").struct.rename_fields(["x", "y", "z"])).unnest("A", "B")

# shape: (1, 4)
# ┌─────┬─────┬─────────┬─────────┐
# │ x   ┆ y   ┆ field_0 ┆ field_1 │
# │ --- ┆ --- ┆ ---     ┆ ---     │
# │ i64 ┆ i64 ┆ i64     ┆ i64     │
# ╞═════╪═════╪═════════╪═════════╡
# │ 1   ┆ 2   ┆ 3       ┆ 4       │
# └─────┴─────┴─────────┴─────────┘

# A.field_1 disappears
df.with_columns(pl.col("A").struct.rename_fields(["x"])).unnest("A", "B")

# shape: (1, 3)
# ┌─────┬─────────┬─────────┐
# │ x   ┆ field_0 ┆ field_1 │
# │ --- ┆ ---     ┆ ---     │
# │ i64 ┆ i64     ┆ i64     │
# ╞═════╪═════════╪═════════╡
# │ 1   ┆ 3       ┆ 4       │
# └─────┴─────────┴─────────┘

df.with_columns(pl.col("A").struct.rename_fields([])).unnest("A", "B")

# shape: (1, 3)
# ┌──────┬─────────┬─────────┐
# │      ┆ field_0 ┆ field_1 │
# │ ---  ┆ ---     ┆ ---     │
# │ null ┆ i64     ┆ i64     │
# ╞══════╪═════════╪═════════╡
# │ null ┆ 3       ┆ 4       │
# └──────┴─────────┴─────────┘

Expected behavior

I'm not sure.

Installed versions

--------Version info---------
Polars:      0.17.15
Index type:  UInt32
Python:      3.10.9 (main, Jan 14 2023, 16:56:26)

----Optional dependencies----
numpy:       1.23.5
pandas:      2.0.1
pyarrow:     9.0.0
connectorx:  <not installed>
deltalake:   <not installed>
fsspec:      2022.8.2
matplotlib:  3.5.1
xlsx2csv:    0.8
xlsxwriter:  3.0.8
@cmdlineluser cmdlineluser added bug Something isn't working python Related to Python Polars labels May 26, 2023
@stinodego
Copy link
Member

This should definitely throw an error. Good catch.

@stinodego stinodego added the needs triage Awaiting prioritization by a maintainer label Jan 13, 2024
@stinodego stinodego added A-dtype-struct Area: struct data type P-low Priority: low A-input-parsing Area: parsing input arguments and removed needs triage Awaiting prioritization by a maintainer labels Jan 22, 2024
@stinodego
Copy link
Member

stinodego commented Jan 22, 2024

Copy-pasteable example of the case that should error:

import polars as pl

df = pl.DataFrame({"A": [{"p": 1, "q": 2}]})

result = df.select(pl.col("A").struct.rename_fields(["x"])).unnest("A")
print(result)
shape: (1, 1)
┌─────┐
│ x   │
│ --- │
│ i64 │
╞═════╡
│ 1   │
└─────┘

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-dtype-struct Area: struct data type A-input-parsing Area: parsing input arguments bug Something isn't working P-low Priority: low python Related to Python Polars
Projects
Status: Ready
Development

Successfully merging a pull request may close this issue.

2 participants