Skip to content

Rework starargs with union argument #19651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

Rework starargs with union argument #19651

wants to merge 2 commits into from
+431 −1

Conversation

randolf-scholz
Copy link
Contributor

@randolf-scholz randolf-scholz commented Aug 13, 2025 •
edited
Loading

Makes *args inference smarter by special casing UnionType

  1. *(tuple[A, B, C] | tuple[None|None|None]) gets treated like tuple[A | None, B | None, C | None] inside ArgTypeExpander (applies if all union member are fixed size tuples of equal length
  2. *(Iterable[X] | Iterable[Y]) gets treated like *Iterable[X | Y] inside ArgTypeExpander (applies if ① is not triggered)

This gives some better inference in some cases:

x: list[str | None] | list[str]
reveal_type([*x])      # master: list[Any]        PR: list[str | None]
x2: tuple[int, int] | tuple[None, None]
reveal_type( (*x2,) )  # master: tuple[Any, ...]  PR: tuple[int | None, int | None]
x3: tuple[int, int] | tuple[None, None, None]
reveal_type( (*x3,) )  # master: tuple[Any, ...]  PR: tuple[int | None, ...]

See added unit tests for more examples.

See Also: #19650, cc @hauntsaninja

Dev notes:

  • If we detect that the union has non-tuple elements or tuples of different sizes, we upcast-reinterpret every union member as some Iterable[T], then apply each union member, and then return the union of the results.
    Handling unions of finite size tuples with different sizes in infeasible, due to combinatoric explosion1
    -visit_tuple_expr: added check enables (*x2,) = tuple[int | None, int | None] when x2=tuple[int, int] | tuple[None, None]
  • map_actuals_to_formals added special case for union of same sized tuples.

Footnotes

  1. consider f(*x1, *x2, ..., *xn). If each $x_k$ is comprised of a union of $m_k$ differently sized tuples, then there are $m_1⋅m_2⋅…⋅m_k$ possible paths. ↩

@randolf-scholz randolf-scholz force-pushed the fix_list_comprehension branch from 7146a29 to 049afbc Compare August 13, 2025 12:38
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@randolf-scholz
Copy link
Contributor Author

Hm there is still some issue to be solved with Unpack. The flatten_nested_tuples method helps./

I think one could determine the correct upcast by using the solver, but I discovered some inconsistencies: #19652

@randolf-scholz
Copy link
Contributor Author

This really does catch quite a few false negatives, for example:

Repro of homeassistant/components/govee_light_local https://mypy-play.net/?mypy=latest&python=3.12&gist=3bf7d56a2bfbc1fc23be62ef521dc613

def set_rgb_color(red: int, green: int, blue: int) -> None: ...

_last_color_state: tuple[
    str | None,
    int | None,
    tuple[int, int, int] | tuple[int | None] | None,
]

color_mode, brightness, color = _last_color_state
if color:
    set_rgb_color(*color)  # MASTER: OK, PR: raises [arg-type]

@github-actions GitHub Actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

tornado (https://github.com/tornadoweb/tornado)
+ tornado/routing.py:355: error: Argument 2 to "Rule" has incompatible type "*Union[list[Any], tuple[Any], tuple[Any, dict[str, Any]], tuple[Any, dict[str, Any], str]]"; expected "Optional[dict[str, Any]]"  [arg-type]
+ tornado/routing.py:355: error: Argument 2 to "Rule" has incompatible type "*Union[list[Any], tuple[Any], tuple[Any, dict[str, Any]], tuple[Any, dict[str, Any], str]]"; expected "Optional[str]"  [arg-type]
+ tornado/routing.py:357: error: Argument 1 to "Rule" has incompatible type "*Union[list[Any], tuple[Union[str, Matcher], Any], tuple[Union[str, Matcher], Any, dict[str, Any]], tuple[Union[str, Matcher], Any, dict[str, Any], str]]"; expected "Matcher"  [arg-type]
+ tornado/routing.py:357: error: Argument 1 to "Rule" has incompatible type "*Union[list[Any], tuple[Union[str, Matcher], Any], tuple[Union[str, Matcher], Any, dict[str, Any]], tuple[Union[str, Matcher], Any, dict[str, Any], str]]"; expected "Optional[dict[str, Any]]"  [arg-type]
+ tornado/routing.py:357: error: Argument 1 to "Rule" has incompatible type "*Union[list[Any], tuple[Union[str, Matcher], Any], tuple[Union[str, Matcher], Any, dict[str, Any]], tuple[Union[str, Matcher], Any, dict[str, Any], str]]"; expected "Optional[str]"  [arg-type]

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/web_urldispatcher.py:671:16: error: Exception type must be derived from BaseException (or be a tuple of exception classes)  [misc]

core (https://github.com/home-assistant/core)
+ homeassistant/components/govee_light_local/light.py:216: error: Argument 2 to "set_rgb_color" of "GoveeLocalApiCoordinator" has incompatible type "*tuple[int, int, int] | tuple[int | None]"; expected "int"  [arg-type]
+ homeassistant/components/govee_light_local/light.py:218: error: Argument 2 to "set_temperature" of "GoveeLocalApiCoordinator" has incompatible type "*tuple[int, int, int] | tuple[int | None]"; expected "int"  [arg-type]

comtypes (https://github.com/enthought/comtypes)
- comtypes/_memberspec.py:85: error: Unused "type: ignore" comment  [unused-ignore]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/test.py:436: error: Argument 2 to "add_file" of "FileMultiDict" has incompatible type "*Union[tuple[IO[bytes], str], tuple[IO[bytes], str, str]]"; expected "Optional[str]"  [arg-type]

dd-trace-py (https://github.com/DataDog/dd-trace-py)
+ ddtrace/llmobs/_integrations/bedrock_agents.py:237: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "type[BaseException]"  [arg-type]
+ ddtrace/llmobs/_integrations/bedrock_agents.py:237: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "BaseException"  [arg-type]
+ ddtrace/llmobs/_integrations/bedrock_agents.py:286: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "type[BaseException]"  [arg-type]
+ ddtrace/llmobs/_integrations/bedrock_agents.py:286: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "BaseException"  [arg-type]
+ ddtrace/llmobs/_experiment.py:367: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "type[BaseException]"  [arg-type]
+ ddtrace/llmobs/_experiment.py:367: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "BaseException"  [arg-type]

dedupe (https://github.com/dedupeio/dedupe)
+ dedupe/core.py:196: error: Argument 1 to "zip" has incompatible type "*tuple[int, Mapping[str, Any]] | tuple[str, Mapping[str, Any]]"; expected "Iterable[str]"  [arg-type]

materialize (https://github.com/MaterializeInc/materialize)
+ misc/python/materialize/mzcompose/composition.py:382: error: Argument 1 to "Popen" has incompatible type "list[object]"; expected "str | bytes | PathLike[str] | PathLike[bytes] | Sequence[str | bytes | PathLike[str] | PathLike[bytes]]"  [arg-type]
+ misc/python/materialize/mzcompose/composition.py:433: error: Not all union combinations were tried because there are too many unions  [misc]
+ misc/python/materialize/mzcompose/composition.py:434: error: Argument 1 to "run" has incompatible type "list[object]"; expected "str | bytes | PathLike[str] | PathLike[bytes] | Sequence[str | bytes | PathLike[str] | PathLike[bytes]]"  [arg-type]

bokeh (https://github.com/bokeh/bokeh)
+ src/bokeh/server/tornado.py: note: In member "__init__" of class "BokehTornado":
+ src/bokeh/server/tornado.py:450:41: error: Argument 1 to "append" of "list" has incompatible type "tuple[dict[str, bool | dict[str, ApplicationContext] | str | None] | str | type[RequestHandler] | dict[str, Any], ...]"; expected "tuple[str, type[RequestHandler]] | tuple[str, type[RequestHandler], dict[str, Any]]"  [arg-type]
+ src/bokeh/server/tornado.py:453:37: error: Argument 1 to "append" of "list" has incompatible type "tuple[dict[str, bool | dict[str, ApplicationContext] | str | None] | str | type[RequestHandler] | dict[str, Any], ...]"; expected "tuple[str, type[RequestHandler]] | tuple[str, type[RequestHandler], dict[str, Any]]"  [arg-type]
+ src/bokeh/server/tornado.py: note: At top level:

xarray (https://github.com/pydata/xarray)
+ xarray/tests/test_namedarray.py: note: In member "test_permute_dims" of class "TestNamedArray":
+ xarray/tests/test_namedarray.py:545: error: Argument 1 to "permute_dims" of "NamedArray" has incompatible type "*str | Iterable[Hashable]"; expected "Iterable[Hashable] | EllipsisType"  [arg-type]
+ xarray/tests/test_namedarray.py: note: In class "TestNamedArray":

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@randolf-scholz