Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

result of itertuples gives an error when passed to NamedTuple - incompatible type "_PandasNamedTuple"; expected "NamedTuple" #881

Open
roj516 opened this issue Feb 27, 2024 · 1 comment

Comments

@roj516
Copy link

roj516 commented Feb 27, 2024

Describe the bug

Code that passed the output of itertuples to functions expecting NamedTuple fails in mypy with incompatible type "_PandasNamedTuple"; expected "NamedTuple"
(Maybe related to the fix for issue #834)

To Reproduce

% python3.12 -m venv venv_pandas_stubs
% source venv_pandas_stubs/bin/activate
% pip install --upgrade pip
% pip install mypy pandas pandas-stubs

Collecting mypy
  Downloading mypy-1.8.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (1.8 kB)
Collecting pandas
  Downloading pandas-2.2.1-cp312-cp312-macosx_11_0_arm64.whl.metadata (19 kB)
Collecting pandas-stubs
  Downloading pandas_stubs-2.2.0.240218-py3-none-any.whl.metadata (9.5 kB)

% cat foo.py 

import pandas as pd
from typing import NamedTuple


def process_tuple(t: NamedTuple) -> None:
  pass


def process_dataframe(df: pd.DataFrame) -> None:
  for t in df.itertuples():
    process_tuple(t)

% mypy foo.py                         
foo.py:11: error: Argument 1 to "process_tuple" has incompatible type "_PandasNamedTuple"; expected "NamedTuple"  [arg-type]

% pip install pandas-stubs==2.0.3.230814
% mypy foo.py                           
Success: no issues found in 1 source file

Please complete the following information:

  • OS: MacOS
  • OS Version: 14.3.1
  • python version: 3.12
  • version of type checker: mypy 1.8.0
  • version of installed pandas-stubs: 2.2.0.240218

Additional context
mypy has special casing for NamedTuple - https://mypy.readthedocs.io/en/latest/kinds_of_types.html#named-tuples

@Dr-Irv
Copy link
Collaborator

Dr-Irv commented Feb 27, 2024

So I'm not sure if we'll be able to address this for 2 reasons:

  1. The special feature of a raw NamedTuple “pseudo-class” that mypy has for NamedTuple has the comment "Note that this behavior is highly experimental, non-standard, and may not be supported by other type checkers and IDEs."
  2. The python typing system doesn't really support having a subclass of NamedTuple with undefined fields, which is what we would need to return in this case.

From a static typing perspective, we know we are returning a kind of "generic" named tuple, but we don't know what the fields are, since it has to work for any DataFrame

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants