Skip to content

[doc] bpo-45680: Disambiguate __getitem__ and __class_getitem__ in the data model. #29389

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Nov 18, 2021
1 change: 1 addition & 0 deletions Doc/library/typing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,7 @@ called :class:`TypeVar`.
def first(l: Sequence[T]) -> T: # Generic function
return l[0]

.. _user-defined-generics:

User-defined generic types
==========================
Expand Down
164 changes: 146 additions & 18 deletions Doc/reference/datamodel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2215,22 +2215,142 @@ case the instance is itself a class.
Emulating generic types
-----------------------

One can implement the generic class syntax as specified by :pep:`484`
(for example ``List[int]``) by defining a special method:
When using :term:`type annotations<annotation>`, it is often useful to
*parameterize* a :term:`generic type` using Python's square-brackets notation.
For example, the annotation ``list[int]`` might be used to signify a
:class:`list` in which all the elements are of type :class:`int`.

.. seealso::

:pep:`484` - Type Hints
Introducing Python's framework for type annotations

:ref:`Generic Alias Types<types-genericalias>`
Documentation for objects representing parameterized generic classes

:ref:`Generics`, :ref:`user-defined generics<user-defined-generics>` and :class:`typing.Generic`
Documentation on how to implement generic classes that can be
parameterized at runtime and understood by static type-checkers.

A class can *generally* only be parameterized if it defines the special
class method ``__class_getitem__()``.

.. classmethod:: object.__class_getitem__(cls, key)

Return an object representing the specialization of a generic class
by type arguments found in *key*.

This method is looked up on the class object itself, and when defined in
the class body, this method is implicitly a class method. Note, this
mechanism is primarily reserved for use with static type hints, other usage
is discouraged.
When defined on a class, ``__class_getitem__()`` is automatically a class
method. As such, there is no need for it to be decorated with
:func:`@classmethod<classmethod>` when it is defined.

.. seealso::

:pep:`560` - Core support for typing module and generic types
The purpose of *__class_getitem__*
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The purpose of :meth:`~object.__class_getitem__` is to allow runtime
parameterization of standard-library generic classes in order to more easily
apply :term:`type hints<type hint>` to these classes.

To implement custom generic classes that can be parameterized at runtime and
understood by static type-checkers, users should either inherit from a standard
library class that already implements :meth:`~object.__class_getitem__`, or
inherit from :class:`typing.Generic`, which has its own implementation of
``__class_getitem__()``.

Custom implementations of :meth:`~object.__class_getitem__` on classes defined
outside of the standard library may not be understood by third-party
type-checkers such as mypy. Using ``__class_getitem__()`` on any class for
purposes other than type hinting is discouraged.


.. _classgetitem-versus-getitem:


*__class_getitem__* versus *__getitem__*
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Usually, the :ref:`subscription<subscriptions>` of an object using square
brackets will call the :meth:`~object.__getitem__` instance method defined on
the object's class. However, if the object being subscribed is itself a class,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be "object being subscribed", or "object being subscripted"?

the class method :meth:`~object.__class_getitem__` may be called instead.
``__class_getitem__()`` should return a :ref:`GenericAlias<types-genericalias>`
object if it is properly defined.

Presented with the :term:`expression` ``obj[x]``, the Python interpreter
follows something like the following process to decide whether
:meth:`~object.__getitem__` or :meth:`~object.__class_getitem__` should be
called::

from inspect import isclass

def subscribe(obj, x):
"""Return the result of the expression `obj[x]`"""

class_of_obj = type(obj)

# If the class of obj defines __getitem__,
# call class_of_obj.__getitem__(obj, x)
if hasattr(class_of_obj, '__getitem__'):
return class_of_obj.__getitem__(obj, x)

# Else, if obj is a class and defines __class_getitem__,
# call obj.__class_getitem__(x)
elif isclass(obj) and hasattr(obj, '__class_getitem__'):
return obj.__class_getitem__(x)

# Else, raise an exception
else:
raise TypeError(
f"'{class_of_obj.__name__}' object is not subscriptable"
)

In Python, all classes are themselves instances of other classes. The class of
a class is known as that class's :term:`metaclass`, and most classes have the
:class:`type` class as their metaclass. :class:`type` does not define
:meth:`~object.__getitem__`, meaning that expressions such as ``list[int]``,
``dict[str, float]`` and ``tuple[str, bytes]`` all result in
:meth:`~object.__class_getitem__` being called::

>>> # list has class "type" as its metaclass, like most classes:
>>> type(list)
<class 'type'>
>>> type(dict) == type(list) == type(tuple) == type(str) == type(bytes)
True
>>> # "list[int]" calls "list.__class_getitem__(int)"
>>> list[int]
list[int]
>>> # list.__class_getitem__ returns a GenericAlias object:
>>> type(list[int])
<class 'types.GenericAlias'>

However, if a class has a custom metaclass that defines
:meth:`~object.__getitem__`, subscribing the class may result in different
behaviour. An example of this can be found in the :mod:`enum` module::

>>> from enum import Enum
>>> class Menu(Enum):
... """A breakfast menu"""
... SPAM = 'spam'
... BACON = 'bacon'
...
>>> # Enum classes have a custom metaclass:
>>> type(Menu)
<class 'enum.EnumMeta'>
>>> # EnumMeta defines __getitem__,
>>> # so __class_getitem__ is not called,
>>> # and the result is not a GenericAlias object:
>>> Menu['SPAM']
<Menu.SPAM: 'spam'>
>>> type(Menu['SPAM'])
<enum 'Menu'>


.. seealso::
:pep:`560` - Core Support for typing module and generic types
Introducing :meth:`~object.__class_getitem__`, and outlining when a
:ref:`subscription<subscriptions>` results in ``__class_getitem__()``
being called instead of :meth:`~object.__getitem__`


.. _callable-types:
Expand Down Expand Up @@ -2330,19 +2450,27 @@ through the object's keys; for sequences, it should iterate through the values.

.. method:: object.__getitem__(self, key)

Called to implement evaluation of ``self[key]``. For sequence types, the
accepted keys should be integers and slice objects. Note that the special
interpretation of negative indexes (if the class wishes to emulate a sequence
type) is up to the :meth:`__getitem__` method. If *key* is of an inappropriate
type, :exc:`TypeError` may be raised; if of a value outside the set of indexes
for the sequence (after any special interpretation of negative values),
:exc:`IndexError` should be raised. For mapping types, if *key* is missing (not
in the container), :exc:`KeyError` should be raised.
Called to implement evaluation of ``self[key]``. For :term:`sequence` types,
the accepted keys should be integers and slice objects. Note that the
special interpretation of negative indexes (if the class wishes to emulate a
:term:`sequence` type) is up to the :meth:`__getitem__` method. If *key* is
of an inappropriate type, :exc:`TypeError` may be raised; if of a value
outside the set of indexes for the sequence (after any special
interpretation of negative values), :exc:`IndexError` should be raised. For
:term:`mapping` types, if *key* is missing (not in the container),
:exc:`KeyError` should be raised.

.. note::

:keyword:`for` loops expect that an :exc:`IndexError` will be raised for
illegal indexes to allow proper detection of the end of the sequence.

.. note::

:keyword:`for` loops expect that an :exc:`IndexError` will be raised for illegal
indexes to allow proper detection of the end of the sequence.
When :ref:`subscripting<subscriptions>` a *class*, the special
class method :meth:`~object.__class_getitem__` may be called instead of
``__getitem__()``. See :ref:`classgetitem-versus-getitem` for more
details.


.. method:: object.__setitem__(self, key, value)
Expand Down