Skip to content

[3.13] gh-123523: Rework typing documentation for generators and coroutines, and link to it from collections.abc docs (GH-123544) #123790

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 6, 2024
Merged

[3.13] gh-123523: Rework typing documentation for generators and coroutines, and link to it from collections.abc docs (GH-123544) #123790

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions Doc/library/collections.abc.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,9 @@ Collections Abstract Base Classes -- Detailed Descriptions

ABC for classes that provide the :meth:`~object.__call__` method.

See :ref:`annotating-callables` for details on how to use
:class:`!Callable` in type annotations.

.. class:: Iterable

ABC for classes that provide the :meth:`~container.__iter__` method.
Expand Down Expand Up @@ -253,6 +256,9 @@ Collections Abstract Base Classes -- Detailed Descriptions
:meth:`~generator.send`,
:meth:`~generator.throw` and :meth:`~generator.close` methods.

See :ref:`annotating-generators-and-coroutines`
for details on using :class:`!Generator` in type annotations.

.. versionadded:: 3.5

.. class:: Sequence
Expand Down Expand Up @@ -331,6 +337,11 @@ Collections Abstract Base Classes -- Detailed Descriptions
Using ``isinstance(gencoro, Coroutine)`` for them will return ``False``.
Use :func:`inspect.isawaitable` to detect them.

See :ref:`annotating-generators-and-coroutines`
for details on using :class:`!Coroutine` in type annotations.
The variance and order of type parameters correspond to those of
:class:`Generator`.

.. versionadded:: 3.5

.. class:: AsyncIterable
Expand All @@ -352,6 +363,9 @@ Collections Abstract Base Classes -- Detailed Descriptions
ABC for :term:`asynchronous generator` classes that implement the protocol
defined in :pep:`525` and :pep:`492`.

See :ref:`annotating-generators-and-coroutines`
for details on using :class:`!AsyncGenerator` in type annotations.

.. versionadded:: 3.6

.. class:: Buffer
Expand Down
199 changes: 98 additions & 101 deletions Doc/library/typing.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ Annotating callable objects
===========================

Functions -- or other :term:`callable` objects -- can be annotated using
:class:`collections.abc.Callable` or :data:`typing.Callable`.
:class:`collections.abc.Callable` or deprecated :data:`typing.Callable`.
``Callable[[int], str]`` signifies a function that takes a single parameter
of type :class:`int` and returns a :class:`str`.

Expand Down Expand Up @@ -401,7 +401,7 @@ The type of class objects
=========================

A variable annotated with ``C`` may accept a value of type ``C``. In
contrast, a variable annotated with ``type[C]`` (or
contrast, a variable annotated with ``type[C]`` (or deprecated
:class:`typing.Type[C] <Type>`) may accept values that are classes
themselves -- specifically, it will accept the *class object* of ``C``. For
example::
Expand Down Expand Up @@ -441,6 +441,87 @@ For example::
``type[Any]`` is equivalent to :class:`type`, which is the root of Python's
:ref:`metaclass hierarchy <metaclasses>`.


.. _annotating-generators-and-coroutines:

Annotating generators and coroutines
====================================

A generator can be annotated using the generic type
:class:`Generator[YieldType, SendType, ReturnType] <collections.abc.Generator>`.
For example::

def echo_round() -> Generator[int, float, str]:
sent = yield 0
while sent >= 0:
sent = yield round(sent)
return 'Done'

Note that unlike many other generic classes in the standard library,
the ``SendType`` of :class:`~collections.abc.Generator` behaves
contravariantly, not covariantly or invariantly.

The ``SendType`` and ``ReturnType`` parameters default to :const:`!None`::

def infinite_stream(start: int) -> Generator[int]:
while True:
yield start
start += 1

It is also possible to set these types explicitly::

def infinite_stream(start: int) -> Generator[int, None, None]:
while True:
yield start
start += 1

Simple generators that only ever yield values can also be annotated
as having a return type of either
:class:`Iterable[YieldType] <collections.abc.Iterable>`
or :class:`Iterator[YieldType] <collections.abc.Iterator>`::

def infinite_stream(start: int) -> Iterator[int]:
while True:
yield start
start += 1

Async generators are handled in a similar fashion, but don't
expect a ``ReturnType`` type argument
(:class:`AsyncGenerator[YieldType, SendType] <collections.abc.AsyncGenerator>`).
The ``SendType`` argument defaults to :const:`!None`, so the following definitions
are equivalent::

async def infinite_stream(start: int) -> AsyncGenerator[int]:
while True:
yield start
start = await increment(start)

async def infinite_stream(start: int) -> AsyncGenerator[int, None]:
while True:
yield start
start = await increment(start)

As in the synchronous case,
:class:`AsyncIterable[YieldType] <collections.abc.AsyncIterable>`
and :class:`AsyncIterator[YieldType] <collections.abc.AsyncIterator>` are
available as well::

async def infinite_stream(start: int) -> AsyncIterator[int]:
while True:
yield start
start = await increment(start)

Coroutines can be annotated using
:class:`Coroutine[YieldType, SendType, ReturnType] <collections.abc.Coroutine>`.
Generic arguments correspond to those of :class:`~collections.abc.Generator`,
for example::

from collections.abc import Coroutine
c: Coroutine[list[str], str, int] # Some coroutine defined elsewhere
x = c.send('hi') # Inferred type of 'x' is list[str]
async def bar() -> None:
y = await c # Inferred type of 'y' is int

.. _user-defined-generics:

User-defined generic types
Expand Down Expand Up @@ -3318,14 +3399,9 @@ Aliases to built-in types
Deprecated alias to :class:`dict`.

Note that to annotate arguments, it is preferred
to use an abstract collection type such as :class:`Mapping`
to use an abstract collection type such as :class:`~collections.abc.Mapping`
rather than to use :class:`dict` or :class:`!typing.Dict`.

This type can be used as follows::

def count_words(text: str) -> Dict[str, int]:
...

.. deprecated:: 3.9
:class:`builtins.dict <dict>` now supports subscripting (``[]``).
See :pep:`585` and :ref:`types-genericalias`.
Expand All @@ -3335,16 +3411,9 @@ Aliases to built-in types
Deprecated alias to :class:`list`.

Note that to annotate arguments, it is preferred
to use an abstract collection type such as :class:`Sequence` or
:class:`Iterable` rather than to use :class:`list` or :class:`!typing.List`.

This type may be used as follows::

def vec2[T: (int, float)](x: T, y: T) -> List[T]:
return [x, y]

def keep_positives[T: (int, float)](vector: Sequence[T]) -> List[T]:
return [item for item in vector if item > 0]
to use an abstract collection type such as
:class:`~collections.abc.Sequence` or :class:`~collections.abc.Iterable`
rather than to use :class:`list` or :class:`!typing.List`.

.. deprecated:: 3.9
:class:`builtins.list <list>` now supports subscripting (``[]``).
Expand All @@ -3355,8 +3424,8 @@ Aliases to built-in types
Deprecated alias to :class:`builtins.set <set>`.

Note that to annotate arguments, it is preferred
to use an abstract collection type such as :class:`AbstractSet`
rather than to use :class:`set` or :class:`!typing.Set`.
to use an abstract collection type such as :class:`collections.abc.Set`
rather than to use :class:`set` or :class:`typing.Set`.

.. deprecated:: 3.9
:class:`builtins.set <set>` now supports subscripting (``[]``).
Expand Down Expand Up @@ -3552,11 +3621,6 @@ Aliases to container ABCs in :mod:`collections.abc`

Deprecated alias to :class:`collections.abc.Mapping`.

This type can be used as follows::

def get_position_in_index(word_list: Mapping[str, int], word: str) -> int:
return word_list[word]

.. deprecated:: 3.9
:class:`collections.abc.Mapping` now supports subscripting (``[]``).
See :pep:`585` and :ref:`types-genericalias`.
Expand Down Expand Up @@ -3620,14 +3684,9 @@ Aliases to asynchronous ABCs in :mod:`collections.abc`

Deprecated alias to :class:`collections.abc.Coroutine`.

The variance and order of type variables
correspond to those of :class:`Generator`, for example::

from collections.abc import Coroutine
c: Coroutine[list[str], str, int] # Some coroutine defined elsewhere
x = c.send('hi') # Inferred type of 'x' is list[str]
async def bar() -> None:
y = await c # Inferred type of 'y' is int
See :ref:`annotating-generators-and-coroutines`
for details on using :class:`collections.abc.Coroutine`
and ``typing.Coroutine`` in type annotations.

.. versionadded:: 3.5.3

Expand All @@ -3639,40 +3698,9 @@ Aliases to asynchronous ABCs in :mod:`collections.abc`

Deprecated alias to :class:`collections.abc.AsyncGenerator`.

An async generator can be annotated by the generic type
``AsyncGenerator[YieldType, SendType]``. For example::

async def echo_round() -> AsyncGenerator[int, float]:
sent = yield 0
while sent >= 0.0:
rounded = await round(sent)
sent = yield rounded

Unlike normal generators, async generators cannot return a value, so there
is no ``ReturnType`` type parameter. As with :class:`Generator`, the
``SendType`` behaves contravariantly.

The ``SendType`` defaults to :const:`!None`::

async def infinite_stream(start: int) -> AsyncGenerator[int]:
while True:
yield start
start = await increment(start)

It is also possible to set this type explicitly::

async def infinite_stream(start: int) -> AsyncGenerator[int, None]:
while True:
yield start
start = await increment(start)

Alternatively, annotate your generator as having a return type of
either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``::

async def infinite_stream(start: int) -> AsyncIterator[int]:
while True:
yield start
start = await increment(start)
See :ref:`annotating-generators-and-coroutines`
for details on using :class:`collections.abc.AsyncGenerator`
and ``typing.AsyncGenerator`` in type annotations.

.. versionadded:: 3.6.1

Expand Down Expand Up @@ -3754,40 +3782,9 @@ Aliases to other ABCs in :mod:`collections.abc`

Deprecated alias to :class:`collections.abc.Generator`.

A generator can be annotated by the generic type
``Generator[YieldType, SendType, ReturnType]``. For example::

def echo_round() -> Generator[int, float, str]:
sent = yield 0
while sent >= 0:
sent = yield round(sent)
return 'Done'

Note that unlike many other generics in the typing module, the ``SendType``
of :class:`Generator` behaves contravariantly, not covariantly or
invariantly.

The ``SendType`` and ``ReturnType`` parameters default to :const:`!None`::

def infinite_stream(start: int) -> Generator[int]:
while True:
yield start
start += 1

It is also possible to set these types explicitly::

def infinite_stream(start: int) -> Generator[int, None, None]:
while True:
yield start
start += 1

Alternatively, annotate your generator as having a return type of
either ``Iterable[YieldType]`` or ``Iterator[YieldType]``::

def infinite_stream(start: int) -> Iterator[int]:
while True:
yield start
start += 1
See :ref:`annotating-generators-and-coroutines`
for details on using :class:`collections.abc.Generator`
and ``typing.Generator`` in type annotations.

.. deprecated:: 3.9
:class:`collections.abc.Generator` now supports subscripting (``[]``).
Expand Down
Loading