Skip to content

t.Optional[t.Union[...]] leads to incorrect behaviour #19204

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
atharva-kelkar opened this issue Jun 2, 2025 · 5 comments
Closed

t.Optional[t.Union[...]] leads to incorrect behaviour #19204

atharva-kelkar opened this issue Jun 2, 2025 · 5 comments
Labels
bug mypy got something wrong

Comments

@atharva-kelkar
Copy link

atharva-kelkar commented Jun 2, 2025 •
edited
Loading

Bug Report
When using t.Optional with t.Union, mypy checks fail with conditional processing of different types. However, removing t.Optional from the code below leads to a successful check.

To Reproduce

import pandas as pd
import numpy as np
import typing as t

def cast_to_pandas(x: np.ndarray) -> t.Union[pd.Series, pd.DataFrame]:
    if x.ndim == 1:
        return pd.Series(x)
    return pd.DataFrame(x)

def pred(x: t.Optional[t.Union[np.ndarray, pd.Series, pd.DataFrame]] = np.ones(10)) -> None:
    if isinstance(x, np.ndarray):
        x = cast_to_pandas(x)

    print(x.values.ravel)

Expected Behavior
I would have expected this code to pass mypy type-checking because an np.ndarray type is separately cast into pd.Series or pd.DataFrame.

Actual Behavior
Get the following error:

trial.py:14: error: Item "ndarray[Any, Any]" of "ndarray[Any, Any] | Any" has no attribute "values"  [union-attr]
Found 1 error in 1 file

Your Environment
mypy==1.16.0
pandas==2.2.3
numpy==2.2.6

  • Mypy version used: 1.16.0
  • Mypy command-line flags: -p package
  • Mypy configuration options from mypy.ini (and other config files): -
  • Python version used: 3.13.3

Edits: Removed redundant comments.
@atharva-kelkar atharva-kelkar added the bug mypy got something wrong label Jun 2, 2025
@JelleZijlstra
Copy link
Member

I strongly suspect this is because mypy doesn't see types for pandas, and therefore treats pd.Series and pd.DataFrame as Any.

Are you sure the t.Optional part is relevant? Note that Optional is sugar for a union with None.

@atharva-kelkar
Copy link
Author

Hey, thanks for the reply.

mypy doesn't see types for pandas, and therefore treats pd.Series and pd.DataFrame as Any.

I am not sure whether this affects the issue I am seeing. The issue is that mypy does not recognize that the np.ndarray type has been eliminated in the conditional statement and still type-checks for it when t.Optional is present.

I am sure the t.Optional part is somehow relevant (I do not know exactly why though) because you can type-check my code without the t.Optional and all mypy checks are passed.

@hauntsaninja
Copy link
Collaborator

Try installing pandas-stubs. If mypy does not know what pd.Series and pd.DataFrame are it cannot know that np.ndarray has been eliminated

@atharva-kelkar
Copy link
Author

atharva-kelkar commented Jun 3, 2025 •
edited
Loading

Thanks @hauntsaninja, pandas-stubs did do the trick. However, I still do not get why in the case of no pandas-stubs, it works without the t.Optional but not with it.

@hauntsaninja
Copy link
Collaborator

After you assign x = cast(Any, ...), mypy will reset x to the broadest type it has available. If the type is ndarray | Any, you can access .values. If the type is ndarray | Any | None, you cannot access .values

@hauntsaninja hauntsaninja closed this as not planned Won't fix, can't repro, duplicate, stale Jun 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug mypy got something wrong
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@JelleZijlstra @hauntsaninja @atharva-kelkar