Rework starargs with union argument #19651

randolf-scholz · 2025-08-13T12:32:35Z

Makes *args inference smarter by special casing UnionType

*(tuple[A, B, C] | tuple[None|None|None]) gets treated like tuple[A | None, B | None, C | None] inside ArgTypeExpander (applies if all union member are fixed size tuples of equal length
*(Iterable[X] | Iterable[Y]) gets treated like *Iterable[X | Y] inside ArgTypeExpander (applies if ① is not triggered)

This gives some better inference in some cases:

x: list[str | None] | list[str]
reveal_type([*x])      # master: list[Any]        PR: list[str | None]
x2: tuple[int, int] | tuple[None, None]
reveal_type( (*x2,) )  # master: tuple[Any, ...]  PR: tuple[int | None, int | None]
x3: tuple[int, int] | tuple[None, None, None]
reveal_type( (*x3,) )  # master: tuple[Any, ...]  PR: tuple[int | None, ...]

See added unit tests for more examples.

Dev notes:

If we detect that the union has non-tuple elements or tuples of different sizes, we upcast-reinterpret every union member as some Iterable[T], then apply each union member, and then return the union of the results.
Handling unions of finite size tuples with different sizes in infeasible, due to combinatoric explosion¹
map_actuals_to_formals added special case for union of same sized tuples.
ExpressionChecker.check_arg: added some extra logic that applies if the callee_type is UnpackType, but the actual_type got expanded from Unpack_type to something else. Previously, this only worked "by accident" because the expander was guaranteed to return AnyType(TypeOfAny.from_errors) in this case.
ArgTypeExpander:
- Create a NewType alias IterableType for Iteralble[T], and helper functions is_iterable, is_iterable_instance_type, _make_iterable_instance_type.
- added _solve_as_iterable function that uses the solver to interpret a type as Iterable[T]
- added as_iterable_type that applies a few special cases (UnionType, etc. before calling the solver)
- added parse_star_args_type that converts the argument of *args to one of TupleType | IterableType | ParamSpecType | AnyType
visit_tuple_expr: Applies ArgTypeExpander(*parse_star_args_type) to ensure consistent behavior across [*args], {*args} and (*args).

consider f(*x1, *x2, ..., *xn). If each $x_k$ is comprised of a union of $m_k$ differently sized tuples, then there are $m_1⋅m_2⋅…⋅m_k$ possible paths. ↩

randolf-scholz · 2025-08-13T14:19:33Z

Hm there is still some issue to be solved with Unpack. The flatten_nested_tuples method helps./

I think one could determine the correct upcast by using the solver, but I discovered some inconsistencies: #19652

randolf-scholz · 2025-08-13T14:29:28Z

This really does catch quite a few false negatives, for example:

Repro of homeassistant/components/govee_light_local https://mypy-play.net/?mypy=latest&python=3.12&gist=3bf7d56a2bfbc1fc23be62ef521dc613

def set_rgb_color(red: int, green: int, blue: int) -> None: ...

_last_color_state: tuple[
    str | None,
    int | None,
    tuple[int, int, int] | tuple[int | None] | None,
]

color_mode, brightness, color = _last_color_state
if color:
    set_rgb_color(*color)  # MASTER: no error ❌, PR: raises [arg-type] ✅

Repro of xarray/tests/test_namedarray: https://mypy-play.net/?mypy=latest&python=3.12&gist=9999ca2cb89e849978c979c1e4cc9bca

from typing import Iterable, Hashable

type _Dim = Hashable
type _Dims = tuple[_Dim, ...]
type _DimsLike = str | Iterable[_Dim]
def permute_dims(*dim: Iterable[_Dim]) -> None: ...

def test(dims: _DimsLike) -> None:
    permute_dims(*dims)  # master: no error ❌, PR: raises [arg-type] ✅

Repro of src/bokeh/server/tornado.py:450: https://mypy-play.net/?mypy=latest&python=3.12&gist=0df919e6986b3fcbdb9d7c8727fb8d48

from typing import Any

class RequestHandler: ...
type RouteContext = dict[str, Any]

type URLRoutes = list[
    tuple[str, type[RequestHandler]] |
    tuple[str, type[RequestHandler], RouteContext],
]

def demo(
    all_patterns: URLRoutes,
    extra_patterns: URLRoutes,
    toplevel_patterns: URLRoutes,
    prefix: str,
    data: dict[str, Any],
) -> None:
    for p in extra_patterns + toplevel_patterns:
        prefixed_pat = (prefix + p[0], *p[1:], data)
        all_patterns.append(prefixed_pat)  # MASTER: false negative, PR: [arg-type]

randolf-scholz · 2025-08-15T14:01:59Z

I think this is ready for initial review.

mypy/argmap.py

test-data/unit/lib-stub/builtins.pyi

mypy/argmap.py

This reverts commit e8dcf88.

randolf-scholz · 2025-08-16T08:34:17Z

Found 4 more older issues that all get fixed:

Fixes: Variadic *args where args: Union[Tuple[...], Tuple[...], ...] are _never_ checked [False Negative] #17161
Fixes: Unpacking an iterable in a list comprehension leads to the type inferred as list[Any] #15747
Fixes: Custom __iter__ method yields false positive when used with star args (*) #14470
Fixes: Unpacking unions of tuples skips type checking #15969

mypy/argmap.py

github-actions · 2025-08-16T19:42:59Z

Diff from mypy_primer, showing the effect of this PR on open source code:

tornado (https://github.com/tornadoweb/tornado)
+ tornado/routing.py:355: error: Argument 2 to "Rule" has incompatible type "*Union[list[Any], tuple[Any], tuple[Any, dict[str, Any]], tuple[Any, dict[str, Any], str]]"; expected "Optional[dict[str, Any]]"  [arg-type]
+ tornado/routing.py:355: error: Argument 2 to "Rule" has incompatible type "*Union[list[Any], tuple[Any], tuple[Any, dict[str, Any]], tuple[Any, dict[str, Any], str]]"; expected "Optional[str]"  [arg-type]
+ tornado/routing.py:357: error: Argument 1 to "Rule" has incompatible type "*Union[list[Any], tuple[Union[str, Matcher], Any], tuple[Union[str, Matcher], Any, dict[str, Any]], tuple[Union[str, Matcher], Any, dict[str, Any], str]]"; expected "Matcher"  [arg-type]
+ tornado/routing.py:357: error: Argument 1 to "Rule" has incompatible type "*Union[list[Any], tuple[Union[str, Matcher], Any], tuple[Union[str, Matcher], Any, dict[str, Any]], tuple[Union[str, Matcher], Any, dict[str, Any], str]]"; expected "Optional[dict[str, Any]]"  [arg-type]
+ tornado/routing.py:357: error: Argument 1 to "Rule" has incompatible type "*Union[list[Any], tuple[Union[str, Matcher], Any], tuple[Union[str, Matcher], Any, dict[str, Any]], tuple[Union[str, Matcher], Any, dict[str, Any], str]]"; expected "Optional[str]"  [arg-type]

hydpy (https://github.com/hydpy-dev/hydpy)
+ hydpy/core/selectiontools.py:370: error: Argument 1 to "intersection" of "Devices" has incompatible type "*Elements"; expected "Elements"  [arg-type]
+ hydpy/core/pubtools.py:163: error: Argument 1 to "Timegrids" has incompatible type "*Timegrids | Timegrid | tuple[datetime | str | Date, datetime | str | Date, timedelta | str | Period]"; expected "Timegrid"  [arg-type]
+ hydpy/core/pubtools.py:163: error: Argument 1 to "Timegrids" has incompatible type "*Timegrids | Timegrid | tuple[datetime | str | Date, datetime | str | Date, timedelta | str | Period]"; expected "Timegrid | None"  [arg-type]

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/web_urldispatcher.py:671:16: error: Exception type must be derived from BaseException (or be a tuple of exception classes)  [misc]

colour (https://github.com/colour-science/colour)
+ colour/plotting/models.py:1986: error: Argument 2 to "_linear_equation" has incompatible type "*ndarray[tuple[int], dtype[float64]]"; expected "ndarray[tuple[Any, ...], dtype[floating[_16Bit] | floating[_32Bit] | float64]]"  [arg-type]

core (https://github.com/home-assistant/core)
+ homeassistant/components/govee_light_local/light.py:216: error: Argument 2 to "set_rgb_color" of "GoveeLocalApiCoordinator" has incompatible type "*tuple[int, int, int] | tuple[int | None]"; expected "int"  [arg-type]
+ homeassistant/components/govee_light_local/light.py:218: error: Argument 2 to "set_temperature" of "GoveeLocalApiCoordinator" has incompatible type "*tuple[int, int, int] | tuple[int | None]"; expected "int"  [arg-type]

comtypes (https://github.com/enthought/comtypes)
- comtypes/_memberspec.py:85: error: Unused "type: ignore" comment  [unused-ignore]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/test.py:436: error: Argument 2 to "add_file" of "FileMultiDict" has incompatible type "*Union[tuple[IO[bytes], str], tuple[IO[bytes], str, str]]"; expected "Optional[str]"  [arg-type]

dd-trace-py (https://github.com/DataDog/dd-trace-py)
+ ddtrace/llmobs/_integrations/bedrock_agents.py:237: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "type[BaseException]"  [arg-type]
+ ddtrace/llmobs/_integrations/bedrock_agents.py:237: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "BaseException"  [arg-type]
+ ddtrace/llmobs/_integrations/bedrock_agents.py:286: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "type[BaseException]"  [arg-type]
+ ddtrace/llmobs/_integrations/bedrock_agents.py:286: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "BaseException"  [arg-type]
+ ddtrace/llmobs/_experiment.py:367: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "type[BaseException]"  [arg-type]
+ ddtrace/llmobs/_experiment.py:367: error: Argument 1 to "set_exc_info" of "Span" has incompatible type "*tuple[type[BaseException], BaseException, TracebackType] | tuple[None, None, None]"; expected "BaseException"  [arg-type]

dedupe (https://github.com/dedupeio/dedupe)
+ dedupe/core.py:196: error: Argument 1 to "zip" has incompatible type "*tuple[int, Mapping[str, Any]] | tuple[str, Mapping[str, Any]]"; expected "Iterable[str]"  [arg-type]

materialize (https://github.com/MaterializeInc/materialize)
+ misc/python/materialize/mzcompose/composition.py:382: error: Argument 1 to "Popen" has incompatible type "list[object]"; expected "str | bytes | PathLike[str] | PathLike[bytes] | Sequence[str | bytes | PathLike[str] | PathLike[bytes]]"  [arg-type]
+ misc/python/materialize/mzcompose/composition.py:433: error: Not all union combinations were tried because there are too many unions  [misc]
+ misc/python/materialize/mzcompose/composition.py:434: error: Argument 1 to "run" has incompatible type "list[object]"; expected "str | bytes | PathLike[str] | PathLike[bytes] | Sequence[str | bytes | PathLike[str] | PathLike[bytes]]"  [arg-type]

ignite (https://github.com/pytorch/ignite)
+ ignite/engine/engine.py:156: error: Argument 1 to "register_events" of "Engine" has incompatible type "*type[Events]"; expected "list[str] | list[EventEnum]"  [arg-type]
+ ignite/contrib/engines/tbptt.py:120: error: Argument 1 to "register_events" of "Engine" has incompatible type "*type[Tbptt_Events]"; expected "list[str] | list[EventEnum]"  [arg-type]

xarray (https://github.com/pydata/xarray)
+ xarray/tests/test_namedarray.py: note: In member "test_permute_dims" of class "TestNamedArray":
+ xarray/tests/test_namedarray.py:545: error: Argument 1 to "permute_dims" of "NamedArray" has incompatible type "*str | Iterable[Hashable]"; expected "Iterable[Hashable] | EllipsisType"  [arg-type]
+ xarray/tests/test_namedarray.py: note: In class "TestNamedArray":

bokeh (https://github.com/bokeh/bokeh)
+ src/bokeh/server/tornado.py: note: In member "__init__" of class "BokehTornado":
+ src/bokeh/server/tornado.py:450:41: error: Argument 1 to "append" of "list" has incompatible type "tuple[dict[str, bool | dict[str, ApplicationContext] | str | None] | str | type[RequestHandler] | dict[str, Any], ...]"; expected "tuple[str, type[RequestHandler]] | tuple[str, type[RequestHandler], dict[str, Any]]"  [arg-type]
+ src/bokeh/server/tornado.py:453:37: error: Argument 1 to "append" of "list" has incompatible type "tuple[dict[str, bool | dict[str, ApplicationContext] | str | None] | str | type[RequestHandler] | dict[str, Any], ...]"; expected "tuple[str, type[RequestHandler]] | tuple[str, type[RequestHandler], dict[str, Any]]"  [arg-type]
+ src/bokeh/server/tornado.py: note: At top level:

randolf-scholz · 2025-08-16T20:16:13Z

The src/werkzeug/test.py is a case where the PR introduces a false positive: https://mypy-play.net/?mypy=latest&python=3.12&gist=eb790692cdcb803373269187d9ad0a4a

from typing import IO

def add_file(file: str | IO[bytes], filename: str | None = None) -> None: ...

def test(values: tuple[IO[bytes]] | tuple[IO[bytes], str]) -> None:
    add_file(*values)  # master: OK ✅,  PR:  ❌

Because it upcasts the type to typing.Iterable[typing.IO[builtins.bytes] | builtins.str], whereas master treats it like tuple[Any, ...].

So we need to deal with the case when each union member is a finite tuple differently

One option would be to infer tuple[Any, ...] in this case like on master;
A bit better would be to use the equal size combination logic for the minimal length, and then the join for the tail part, so for instance in the example above, tuple[IO[bytes], *tuple[str, ...]] would be more precise than just tuple[Any, ...].
One could even theoretically do something like tuple[IO[bytes], *(tuple[()] | tuple[str])]], but this is outside the current typing spec, as it breaks the assumption that the argument to Unpack is a tuple or a TypeVarTuple.

I think I can do option 2 for now and combine it with the equal size tuple case.

hauntsaninja

thanks for picking this up!

hauntsaninja · 2025-08-18T06:49:34Z

mypy/argmap.py

+            return AnyType(TypeOfAny.from_error)
+        return self._make_iterable_instance_type(sol)
+
+    def as_iterable_type(self, typ: Type) -> IterableType | AnyType:


can we reuse logic from analyze_iterable_item_type_without_expression or similar?

sterliakov · 2025-08-19T01:36:09Z

mypy/argmap.py

+            ret_type=target,
+            fallback=self.context.function_type,
+        )
+        constraints = infer_constraints_for_callable(


Can mypy.infer.infer_function_type_arguments replace this and following line at once?

sterliakov · 2025-08-19T01:47:05Z

mypy/argmap.py

+                actual_type = TupleType(
+                    tuple_args,
+                    # use Iterable[A | B | C] as the fallback type
+                    fallback=Instance(


AFAIC we only use fallback to store builtins.tuple or a NamedTuple class, never anything else, and there might be an implicit assumption elsewhere that fallback is assignable to tuple[Any, ...], for example. Will anything break if you use builtins.tuple Instance here instead?

sterliakov · 2025-08-19T01:55:15Z

mypy/argmap.py

+            return self.as_iterable_type(p_t.upper_bound)
+        elif self.is_iterable(p_t):
+            # TODO: add a 'fast path' (needs measurement) that uses the map_instance_to_supertype
+            #   mechanism? (Only if it works: gh-19662)


#19662 is listed as closed by this PR, is this comment up to date? (the bug in the original implementation that caused #19662 was checking for and actual_type.args - that adds nothing useful and obviously breaks for non-generic subclasses, removing this check should suffice)

sterliakov · 2025-08-19T01:59:45Z

mypy/argmap.py

+
+    iterator = iter(types)
+    typ = next(iterator)
+    if not isinstance(typ, TupleType):


If you refactor L449-L458 into a helper function, this method would look like

... size = _precise_tuple_len(typ) if size is None: return False return all(_precise_tuple_len(t) == size for t in iterator)

sterliakov · 2025-08-19T02:01:38Z

mypy/checkexpr.py

-                if isinstance(tt, TupleType):
-                    if find_unpack_in_list(tt.items) is not None:
+                original_arg_type = self.accept(item.expr, ctx)
+                # convert arg type to one of TupleType, IterableType, AnyType or


sterliakov · 2025-08-19T02:33:14Z

There are a couple more affected/fixed tickets from the scan above: #18320, #16492, #6799

sterliakov · 2025-08-19T02:37:26Z

A bit better would be to use the equal size combination logic for the minimal length, and then the join for the tail part

This sounds good to me if it can be trivially implemented. If not, let's leave that part as-is or use the any fallback (your option 1) and defer to a separate PR, this PR is already quite long.

randolf-scholz force-pushed the fix_list_comprehension branch from 7146a29 to 049afbc Compare August 13, 2025 12:38

This comment has been minimized.

Sign in to view

randolf-scholz marked this pull request as ready for review August 14, 2025 19:00

randolf-scholz marked this pull request as draft August 14, 2025 19:01

randolf-scholz added 4 commits August 15, 2025 09:34

rework starargs with union argument

245cba3

Use typing.Unpack in tests

f2d254c

reworked argmapper with star args

df7b3f0

always use the solver meachnism

5e239a3

randolf-scholz force-pushed the fix_list_comprehension branch from b341226 to 5e239a3 Compare August 15, 2025 08:13

This comment has been minimized.

Sign in to view

randolf-scholz added 3 commits August 15, 2025 11:55

fixed infer_constraints

6cbb237

cleanup

115bf00

Merge branch 'master' into fix_list_comprehension

08db3da

This comment has been minimized.

Sign in to view

randolf-scholz marked this pull request as ready for review August 15, 2025 14:01

randolf-scholz mentioned this pull request Aug 15, 2025

Fix inference when unpacking union type #19650

Open

sterliakov reviewed Aug 15, 2025

View reviewed changes

randolf-scholz added 2 commits August 15, 2025 17:41

address review

fac1e07

Merge branch 'python:master' into fix_list_comprehension

68bc671

This comment has been minimized.

Sign in to view

randolf-scholz mentioned this pull request Aug 15, 2025

make ParamSpecFlavor an IntEnum and ParamSpecType generic in that type. #19670

Closed

check paramspec flavor

e8dcf88

This comment has been minimized.

Sign in to view

randolf-scholz added 2 commits August 15, 2025 19:24

Revert "check paramspec flavor"

fe4289a

This reverts commit e8dcf88.

update parse_star_args_type docstring

0139c12

update parse_star_args_type docstring

109b4f3

This comment has been minimized.

Sign in to view

randolf-scholz commented Aug 16, 2025

View reviewed changes

mypy/argmap.py Outdated Show resolved Hide resolved

randolf-scholz commented Aug 16, 2025

View reviewed changes

mypy/argmap.py Outdated Show resolved Hide resolved

randolf-scholz commented Aug 16, 2025

View reviewed changes

mypy/argmap.py Outdated Show resolved Hide resolved

randolf-scholz commented Aug 16, 2025

View reviewed changes

mypy/argmap.py Outdated Show resolved Hide resolved

randolf-scholz commented Aug 16, 2025

View reviewed changes

mypy/argmap.py Outdated Show resolved Hide resolved

randolf-scholz added 2 commits August 16, 2025 21:26

Merge branch 'master' into fix_list_comprehension

0a6d757

address review

e169d3e

hauntsaninja reviewed Aug 18, 2025

View reviewed changes

sterliakov reviewed Aug 19, 2025

View reviewed changes

github-actions bot mentioned this pull request Aug 19, 2025

Check #19651 sterliakov/mypy-issues#154

Open

Uh oh!

Rework starargs with union argument #19651

Are you sure you want to change the base?

Rework starargs with union argument #19651

Conversation

randolf-scholz commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dev notes:

Footnotes

Uh oh!

This comment has been minimized.

randolf-scholz commented Aug 13, 2025

Uh oh!

randolf-scholz commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

randolf-scholz commented Aug 15, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

randolf-scholz commented Aug 16, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 16, 2025

Uh oh!

randolf-scholz commented Aug 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hauntsaninja left a comment

Choose a reason for hiding this comment

Uh oh!

hauntsaninja Aug 18, 2025

Choose a reason for hiding this comment

Uh oh!

sterliakov Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

sterliakov Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

sterliakov Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

sterliakov Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

sterliakov Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

sterliakov commented Aug 19, 2025

Uh oh!

sterliakov commented Aug 19, 2025

Uh oh!

Uh oh!

randolf-scholz commented Aug 13, 2025 •

edited

Loading

randolf-scholz commented Aug 13, 2025 •

edited

Loading

randolf-scholz commented Aug 16, 2025 •

edited

Loading