Skip to content

ENH: Add a typing protocol for representing nested sequences #19894

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 20, 2021

Conversation

BvB93
Copy link
Member

@BvB93 BvB93 commented Sep 19, 2021

This PR introduces a protocol representing a recursive version of collections.abc.Sequence (going up to arbitrary levels of nesting), thus allowing for the replacement of the older, more verbose union, the latter of which went to 4 levels of nesting.

The last time it was attempted to introduce such a protocol (#18155 (comment)) there were still a couple of detrimental mypy bugs related recursive objects, thus preventing its introduction. Fortunately, most of these issues were fixed in python/mypy#9663, though recursive objects in combinations typevars seems to be a combination that's still broken (see below).

Examples

>>> from __future__ import annotations
>>> from typing import TypeVar, TYPE_CHECKING
>>> import numpy.typing as npt

>>> T = TypeVar("T")

>>> seq_1d_a: list[int]
>>> seq_1d_b = [1]
>>> seq_2d_a: list[list[int]]
>>> seq_2d_b = [[1]]

>>> def func(array_like: npt._NestedSequence[T]) -> T: ...

>>> if TYPE_CHECKING:
...     # The good
...     reveal_type(func(seq_1d_a))  # Revealed type is "builtins.int*"
...
...     # The bad
...     reveal_type(func(seq_1d_b))  # Revealed type is "Any"
...     reveal_type(func(seq_2d_a))  # Argument 1 to "func" has incompatible type "List[List[int]]"; expected "_NestedSequence[<nothing>]"
...     reveal_type(func(seq_2d_b))  # Revealed type is "Any"

@@ -91,7 +91,7 @@ _DTypeLike = Union[
Type[_SCT],
_SupportsDType[dtype[_SCT]],
]
_ArrayLike = _NestedSequence[_SupportsArray[dtype[_SCT]]]
_ArrayLike = _FiniteNestedSequence[_SupportsArray[dtype[_SCT]]]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used with typevars in def func(a: _ArrayLike[_T]) -> NDArray[_T]: ...-esque expressions, so unfortunately we can't use the new fancy protocol here.

Comment on lines -2457 to -2461
@overload
def __pow__(
self: NDArray[Union[bool_, number[Any]]],
other: _RecursiveSequence,
) -> Any: ...
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need a catchall overload for >4D nested sequences anymore as we can now represent arbitrary levels of nesting.

@charris charris merged commit 68034ed into numpy:main Sep 20, 2021
@charris
Copy link
Member

charris commented Sep 20, 2021

Thanks Bas.

@BvB93 BvB93 deleted the recursive-sequence branch September 20, 2021 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants