Skip to content

__getitem__ iterable support for builtins, collections, itertools #8597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 5 commits into from
Closed

__getitem__ iterable support for builtins, collections, itertools #8597

wants to merge 5 commits into from

Conversation

mixilchenko
Copy link

Lots of objects/collections/itertation tools use for cycle or even iter method explicitly.
iter method supports not only Iterable but also instances of classes with only __getitem__ method implemented so stubs for commonly used things like list, dict, filter`, etc could be improved.

In this PR I add this support only for builtins, collections and itertools as I think these modules need this most.

mixilchenko and others added 5 commits August 23, 2022 14:56
@mixilchenko
Introduce IterableOrSupportsIntGetItem protocol
e1d57a0 
Mostly, `iter` builtin, collections, for loop, accept not only Iterable
but also instances with `__getitem__` implemented
Therefore we need some type to replace `Iterable` with in stubs
@mixilchenko
Support builtins iteration with __getitem__
7b4b38c 
- remove `_GetItemIterable` and use `SupportsIntGetItem` for `iter`
- support `__getitem__` iteration for
  - collections
  - iterators
  - objects arguments
@mixilchenko
Support itertools iteration with __getitem__
37dc024
@mixilchenko
Support collections iteration with __getitem__
f923643
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
0e5af3f
@github-actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

discord.py (https://github.com/Rapptz/discord.py)
- discord/http.py:226: note:     def __setitem__(self, slice, Iterable[Embed]) -> None
+ discord/http.py:226: note:     def __setitem__(self, slice, Union[Iterable[Embed], SupportsIntGetItem[Embed]]) -> None
- discord/abc.py:496: error: Argument 1 to "enumerate" has incompatible type "object"; expected "Iterable[<nothing>]"
+ discord/abc.py:496: error: Argument 1 to "enumerate" has incompatible type "object"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"
- discord/ext/commands/flags.py:614: error: Argument 1 to "dict" has incompatible type "List[str]"; expected "Iterable[Tuple[<nothing>, <nothing>]]"
+ discord/ext/commands/flags.py:614: error: Argument 1 to "dict" has incompatible type "List[str]"; expected "Union[Iterable[Tuple[<nothing>, <nothing>]], SupportsIntGetItem[Tuple[<nothing>, <nothing>]]]"
- discord/ext/commands/cog.py:232: error: Argument 1 to "list" has incompatible type "dict_values[str, _BaseCommand]"; expected "Iterable[Command[Any, [VarArg(Any), KwArg(Any)], Any]]"
+ discord/ext/commands/cog.py:232: error: Argument 1 to "list" has incompatible type "dict_values[str, _BaseCommand]"; expected "Union[Iterable[Command[Any, [VarArg(Any), KwArg(Any)], Any]], SupportsIntGetItem[Command[Any, [VarArg(Any), KwArg(Any)], Any]]]"

cloud-init (https://github.com/canonical/cloud-init)
+ cloudinit/config/schema.py:539: error: List or tuple expected as variadic arguments
+ cloudinit/config/schema.py:541: error: Argument 1 to "filter" has incompatible type "Callable[[Any], bool]"; expected "Callable[[Any], TypeGuard[bool]]"
+ cloudinit/config/schema.py:541: error: Value of type "bool" is not indexable

PyGithub (https://github.com/PyGithub/PyGithub)
+ tests/Connection.py:67: error: Argument 1 to "chain" has incompatible type "*Tuple[Tuple[Type[ReplayingConnection], str], Tuple[str, str]]"; expected "Union[Iterable[str], SupportsIntGetItem[str]]"

operator (https://github.com/canonical/operator)
+ ops/storage.py:113: error: Argument 2 to "execute" of "Connection" has incompatible type "Tuple[str, bytes]"; expected "Union[SupportsLenAndGetItem[Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]], Mapping[str, Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]]]"
+ ops/storage.py:125: error: Argument 2 to "execute" of "Cursor" has incompatible type "Tuple[str]"; expected "Union[SupportsLenAndGetItem[Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]], Mapping[str, Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]]]"
+ ops/storage.py:136: error: Argument 2 to "execute" of "Connection" has incompatible type "Tuple[str]"; expected "Union[SupportsLenAndGetItem[Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]], Mapping[str, Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]]]"
+ ops/storage.py:152: error: Argument 2 to "execute" of "Connection" has incompatible type "Tuple[str, str, str]"; expected "Union[SupportsLenAndGetItem[Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]], Mapping[str, Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]]]"
+ ops/storage.py:161: error: Argument 2 to "execute" of "Connection" has incompatible type "Tuple[str, str, str]"; expected "Union[SupportsLenAndGetItem[Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]], Mapping[str, Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]]]"
+ ops/storage.py:179: error: Argument 2 to "execute" of "Connection" has incompatible type "Tuple[str]"; expected "Union[SupportsLenAndGetItem[Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]], Mapping[str, Union[Union[str, Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData, PickleBuffer]], int, float, None], Any]]]"

jax (https://github.com/google/jax)
+ jax/experimental/maps.py:1798: error: Need type annotation for "defined_axes" (hint: "defined_axes: Set[<type>] = ...")  [var-annotated]
+ jax/experimental/maps.py:1798: error: Argument 1 to "set" has incompatible type "Dict[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]

xarray (https://github.com/pydata/xarray)
+ xarray/core/indexes.py:1400: error: Need type annotation for "common_names" (hint: "common_names: Set[<type>] = ...")  [var-annotated]
+ xarray/core/indexes.py:1400: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/plot/utils.py:417: error: Argument 1 to "set" has incompatible type "DataArrayCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:1719: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:1726: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:2795: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:2795: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[None], SupportsIntGetItem[None]]"  [arg-type]
+ xarray/core/dataset.py:2814: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:3283: error: Need type annotation for "sdims" (hint: "sdims: Set[<type>] = ...")  [var-annotated]
+ xarray/core/dataset.py:3284: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:3537: error: Need type annotation for "old"  [var-annotated]
+ xarray/core/dataset.py:3539: error: Argument 1 to "zip" has incompatible type "Dict[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:4008: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:4018: error: Argument 1 to "list" has incompatible type "Dict[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:4070: error: Argument 1 to "set" has incompatible type "Dict[Hashable, IndexVariable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:4104: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:4148: error: Argument 1 to "set" has incompatible type "Dict[Hashable, IndexVariable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:4512: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:4604: error: Argument 1 to "list" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:4640: error: Need type annotation for "k"  [var-annotated]
+ xarray/core/dataset.py:4641: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:4641: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:4641: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[None], SupportsIntGetItem[None]]"  [arg-type]
+ xarray/core/dataset.py:4808: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:5097: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:5550: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:5840: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:5843: error: Argument 1 to "list" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:6095: error: Need type annotation for "columns" (hint: "columns: List[<type>] = ...")  [var-annotated]
+ xarray/core/dataset.py:6095: error: Argument 1 to "list" has incompatible type "Dict[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:6925: error: Need type annotation for "coord_names" (hint: "coord_names: Set[<type>] = ...")  [var-annotated]
+ xarray/core/dataset.py:6925: error: Argument 1 to "set" has incompatible type "DatasetCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:7753: error: Need type annotation for "pad_dims" (hint: "pad_dims: Set[<type>] = ...")  [var-annotated]
+ xarray/core/dataset.py:7753: error: Argument 1 to "set" has incompatible type "Mapping[Hashable, Union[int, Tuple[int, int]]]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:8292: error: Need type annotation for "preserved_dims" (hint: "preserved_dims: List[<type>] = ...")  [var-annotated]
+ xarray/core/dataset.py:8292: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataset.py:8292: error: Argument 1 to "set" has incompatible type "List[Hashable]"; expected "Union[Iterable[None], SupportsIntGetItem[None]]"  [arg-type]
+ xarray/core/dataset.py:8406: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataarray.py:906: error: Argument 1 to "set" has incompatible type "DataArrayCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataarray.py:906: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/dataarray.py:906: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[None], SupportsIntGetItem[None]]"  [arg-type]
+ xarray/core/coordinates.py:227: error: Need type annotation for "coord_names" (hint: "coord_names: Set[<type>] = ...")  [var-annotated]
+ xarray/core/coordinates.py:227: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/coordinates.py:297: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/coordinates.py:377: error: Argument 1 to "set" has incompatible type "Dict[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/coordinates.py:433: error: Need type annotation for "idx_coord_names" (hint: "idx_coord_names: Set[<type>] = ...")  [var-annotated]
+ xarray/core/coordinates.py:433: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/concat.py:425: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/concat.py:470: error: Need type annotation for "dim_names" (hint: "dim_names: Set[<type>] = ...")  [var-annotated]
+ xarray/core/concat.py:470: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/concat.py:595: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/computation.py:442: error: Need type annotation for "coord_names" (hint: "coord_names: Set[<type>] = ...")  [var-annotated]
+ xarray/core/computation.py:442: error: Argument 1 to "set" has incompatible type "Mapping[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/computation.py:1571: error: Need type annotation for "all_dims" (hint: "all_dims: List[<type>] = ...")  [var-annotated]
+ xarray/core/computation.py:1571: error: Argument 1 to "list" has incompatible type "Dict[Hashable, Optional[Any]]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/accessor_str.py:579: error: Incompatible return value type (got "Union[DataArray, Any]", expected "T_DataArray")  [return-value]
+ xarray/testing.py:353: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/testing.py:361: error: Argument 1 to "set" has incompatible type "Dict[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/parallel.py:46: error: Need type annotation for "dim"  [var-annotated]
+ xarray/core/parallel.py:46: error: Argument 1 to "set" has incompatible type "Mapping[Hashable, Tuple[int, ...]]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/parallel.py:371: error: Need type annotation for "template_indexes" (hint: "template_indexes: Set[<type>] = ...")  [var-annotated]
+ xarray/core/parallel.py:371: error: Argument 1 to "set" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/weighted.py:199: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/rolling.py:974: error: Argument 1 to "set" has incompatible type "DataArrayCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/core/rolling.py:974: error: Argument 1 to "set" has incompatible type "DatasetCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_indexes.py:440: error: Argument 1 to "list" has incompatible type "Dict[Hashable, Index]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_groupby.py:682: error: Need type annotation for "actual1"  [var-annotated]
+ xarray/tests/test_groupby.py:682: error: Argument 1 to "zip" has incompatible type "DatasetGroupBy"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_dataset.py:678: error: Argument 1 to "list" has incompatible type "Frozen[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_dataset.py:728: error: Argument 1 to "list" has incompatible type "DatasetCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_dataset.py:3926: error: Argument 1 to "list" has incompatible type "Frozen[Hashable, Variable]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_dataarray.py:266: error: Argument 1 to "tuple" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_dataarray.py:883: error: Argument 1 to "list" has incompatible type "DataArrayCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_dataarray.py:1336: error: Argument 1 to "list" has incompatible type "DataArrayCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_dataarray.py:6024: error: Need type annotation for "orginal_key"  [var-annotated]
+ xarray/tests/test_dataarray.py:6024: error: Argument 2 to "zip" has incompatible type "DataArrayCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_dataarray.py:6100: error: Need type annotation for "orginal_key"  [var-annotated]
+ xarray/tests/test_dataarray.py:6100: error: Argument 2 to "zip" has incompatible type "DataArrayCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_concat.py:738: error: Argument 1 to "list" has incompatible type "DataArrayCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_concat.py:741: error: Argument 1 to "list" has incompatible type "DataArrayCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_concat.py:802: error: Need type annotation for "act"  [var-annotated]
+ xarray/tests/test_concat.py:802: error: Need type annotation for "exp"  [var-annotated]
+ xarray/tests/test_concat.py:802: error: Argument 1 to "zip" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_concat.py:802: error: Argument 2 to "zip" has incompatible type "Frozen[Hashable, int]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_concat.py:807: error: Argument 1 to "zip" has incompatible type "DatasetCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ xarray/tests/test_concat.py:807: error: Argument 2 to "zip" has incompatible type "DatasetCoordinates"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]

flake8 (https://github.com/pycqa/flake8)
+ src/flake8/plugins/finder.py:27: error: Need type annotation for "FLAKE8_GROUPS"
+ src/flake8/plugins/finder.py:27: error: Argument 1 to "frozenset" has incompatible type "Tuple[str, str]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"
+ tests/unit/plugins/finder_test.py:454: error: Argument 1 to "frozenset" has incompatible type "Tuple[str, str, str]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"
+ tests/unit/plugins/finder_test.py:468: error: Argument 1 to "frozenset" has incompatible type "Tuple[str, str, str]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"

zulip (https://github.com/zulip/zulip)
+ zerver/lib/recipient_users.py:49: error: Need type annotation for "user_ids" (hint: "user_ids: Set[<type>] = ...")  [var-annotated]
+ zerver/lib/subscription_info.py:457: error: Argument 1 to "set" has incompatible type "Dict[int, RawStreamDict]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ zerver/lib/subscription_info.py:457: error: Unsupported operand types for - ("Set[<nothing>]" and "Set[int]")  [operator]
+ zerver/views/development/registration.py:33: error: Argument 1 to "choice" of "Random" has incompatible type "str"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zerver/views/development/registration.py:34: error: Argument 1 to "choice" of "Random" has incompatible type "str"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zerver/data_import/import_util.py:202: error: Argument 1 has incompatible type "List[str]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:467: error: Unsupported left operand type for + (<nothing>)  [operator]
+ zilencer/management/commands/populate_db.py:467: error: Argument 1 has incompatible type "List[str]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:471: error: Argument 1 has incompatible type "List[str]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:473: error: Argument 1 has incompatible type "List[str]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:475: error: Argument 1 has incompatible type "List[str]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:477: error: Argument 1 has incompatible type "List[str]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:901: error: Unsupported left operand type for + (<nothing>)  [operator]
+ zilencer/management/commands/populate_db.py:901: error: Argument 1 has incompatible type "List[str]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:905: error: Argument 1 has incompatible type "List[str]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:1068: error: Argument 1 has incompatible type "List[int]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:1074: error: Argument 1 has incompatible type "Sequence[Sequence[int]]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:1078: error: Argument 1 has incompatible type "List[int]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:1081: error: Need type annotation for "sender_id"  [var-annotated]
+ zilencer/management/commands/populate_db.py:1081: error: Argument 1 has incompatible type "List[int]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:1091: error: <nothing> has no attribute "user_profile"  [attr-defined]
+ zilencer/management/commands/populate_db.py:1092: error: Argument 1 has incompatible type "List[Any]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:1094: error: Argument 1 has incompatible type "List[str]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]
+ zilencer/management/commands/populate_db.py:1170: error: Need type annotation for "num_emojis"  [var-annotated]
+ zilencer/management/commands/populate_db.py:1170: error: Argument 1 has incompatible type "List[int]"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]

black (https://github.com/psf/black)
+ src/black/handle_ipynb_magics.py:19:22: error: Need type annotation for "TRANSFORMED_MAGICS"
+ src/black/handle_ipynb_magics.py:20:5: error: Argument 1 to "frozenset" has incompatible type "Tuple[str, str, str, str]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"
+ src/black/handle_ipynb_magics.py:27:20: error: Need type annotation for "TOKENS_TO_IGNORE"
+ src/black/handle_ipynb_magics.py:28:5: error: Argument 1 to "frozenset" has incompatible type "Tuple[str, str, str, str, str, str, str]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"
+ src/black/handle_ipynb_magics.py:38:22: error: Need type annotation for "PYTHON_CELL_MAGICS"
+ src/black/handle_ipynb_magics.py:39:5: error: Argument 1 to "frozenset" has incompatible type "Tuple[str, str, str, str, str, str, str]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"

materialize (https://github.com/MaterializeInc/materialize)
+ misc/python/materialize/util.py:23: error: Argument 1 has incompatible type "str"; expected "SupportsLenAndGetItem[<nothing>]"
+ misc/python/materialize/zippy/framework.py:124: error: Argument 1 has incompatible type "List[Type[Action]]"; expected "SupportsLenAndGetItem[<nothing>]"
+ misc/python/materialize/zippy/table_actions.py:35: error: Need type annotation for "has_index"
+ misc/python/materialize/zippy/table_actions.py:35: error: Argument 1 has incompatible type "List[bool]"; expected "SupportsLenAndGetItem[<nothing>]"
+ misc/python/materialize/zippy/table_actions.py:68: error: Need type annotation for "table"
+ misc/python/materialize/zippy/table_actions.py:68: error: Argument 1 has incompatible type "List[TableExists]"; expected "SupportsLenAndGetItem[<nothing>]"
+ misc/python/materialize/zippy/table_actions.py:87: error: Need type annotation for "table"
+ misc/python/materialize/zippy/table_actions.py:87: error: Argument 1 has incompatible type "List[TableExists]"; expected "SupportsLenAndGetItem[<nothing>]"
+ misc/python/materialize/zippy/kafka_actions.py:71: error: Argument 1 has incompatible type "List[Envelope]"; expected "SupportsLenAndGetItem[<nothing>]"
+ misc/python/materialize/zippy/kafka_actions.py:112: error: Need type annotation for "topic"
+ misc/python/materialize/zippy/kafka_actions.py:112: error: Argument 1 has incompatible type "List[TopicExists]"; expected "SupportsLenAndGetItem[<nothing>]"
+ misc/python/materialize/zippy/source_actions.py:39: error: Need type annotation for "topic"
+ misc/python/materialize/zippy/source_actions.py:39: error: Argument 1 has incompatible type "List[TopicExists]"; expected "SupportsLenAndGetItem[<nothing>]"
+ misc/python/materialize/zippy/view_actions.py:49: error: Need type annotation for "has_index"
+ misc/python/materialize/zippy/view_actions.py:49: error: Argument 1 has incompatible type "List[bool]"; expected "SupportsLenAndGetItem[<nothing>]"
+ misc/python/materialize/zippy/view_actions.py:96: error: Need type annotation for "view"
+ misc/python/materialize/zippy/view_actions.py:96: error: Argument 1 has incompatible type "List[ViewExists]"; expected "SupportsLenAndGetItem[<nothing>]"
+ misc/python/materialize/zippy/sink_actions.py:39: error: Need type annotation for "source_view"
+ misc/python/materialize/zippy/sink_actions.py:39: error: Argument 1 has incompatible type "List[ViewExists]"; expected "SupportsLenAndGetItem[<nothing>]"

vision (https://github.com/pytorch/vision)
+ torchvision/prototype/datasets/_builtin/mnist.py:400: error: Argument 1 to "zip" has incompatible type "Tuple[str, str, str, str, str]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ torchvision/prototype/datasets/_builtin/mnist.py:405: error: Argument 1 to "zip" has incompatible type "Tuple[str, str]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ torchvision/prototype/datasets/_builtin/imagenet.py:152: error: Argument 1 to "zip" has incompatible type "Tuple[str, str]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"  [arg-type]
+ torchvision/prototype/datasets/_builtin/imagenet.py:152: error: Argument 2 to "zip" has incompatible type "Union[Tuple[Label, str], Tuple[None, None]]"; expected "Union[Iterable[None], SupportsIntGetItem[None]]"  [arg-type]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/datastructures.pyi:51: error: Argument 1 of "extend" is incompatible with supertype "list"; supertype defines the argument type as "Union[Iterable[V], SupportsIntGetItem[V]]"  [override]
+ src/werkzeug/datastructures.pyi:51: note: This violates the Liskov substitution principle
+ src/werkzeug/datastructures.pyi:51: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides
+ src/werkzeug/security.py:24: error: Argument 1 to "choice" has incompatible type "str"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]

manticore (https://github.com/trailofbits/manticore)
+ manticore/core/manticore.py:676: error: Need type annotation for "state_id"
+ manticore/core/manticore.py:676: error: Argument 1 has incompatible type "List[Any]"; expected "SupportsLenAndGetItem[<nothing>]"
+ manticore/platforms/linux.py:249: error: Argument 1 to "max" has incompatible type "Dict[int, FdTableEntry]"; expected "Union[Iterable[<nothing>], SupportsIntGetItem[<nothing>]]"

cwltool (https://github.com/common-workflow-language/cwltool)
+ cwltool/utils.py: note: In function "random_outdir":
+ cwltool/utils.py:240:28: error: Argument 1 has incompatible type "str"; expected "SupportsLenAndGetItem[<nothing>]"  [arg-type]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/computation/ops.py:302: error: Need type annotation for "_cmp_ops_dict"  [var-annotated]
+ pandas/core/computation/ops.py:302: error: Argument 2 to "zip"

... (truncated 295 lines) ...

@AlexWaygood
Copy link
Member

Thanks, but we discussed this in depth in #7813. My opinion has not changed since I wrote #7813 (comment).

I'm guessing you came here from the recent discussion on discuss.python.org. Thanks for trying to help fix the problem, but the mypy_primer output confirms that trying to fix this in the stubs would be disastrous, as it would mean that all users of these classes would have to update their annotations, and Iterable would essentially become useless for type annotations. I don't think fixing this problem justifies such a huge breaking change to the type system.

I think if this is to be fixed, it needs to be done by adding special-casing to type checkers in some way so that they understand that classes with __getitem__ methods are iterable, even if they don't have __iter__ methods.

I'm sorry, but I just don't think this PR is the way to go :)

@anntzer anntzer mentioned this pull request Apr 30, 2024
Open
@hauntsaninja hauntsaninja mentioned this pull request Jul 9, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mixilchenko @AlexWaygood