From 25932fe39505a5ca072cc23e9bed15d10e2c0bcd Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Sun, 17 Aug 2025 07:23:07 -0700 Subject: [PATCH 1/5] spec: Rewrite TypedDict spec This is an edit of the TypedDict spec for clarity and flow. My goal was to unify the pieces of the spec that derive from the various PEPs into a coherent whole. I removed excessive examples and motivations: the spec should specify, not justify. The length of the spec chapter is reduced by more than half. This change is on top of #2068 (adding PEP 728). The general approach I took is to first define the kinds of TypedDicts that can exist, then explain the syntax for defining TypedDicts, then discuss other aspects of TypedDict types. I introduce some new terminology around PEP 728 to make it easier to talk about the different kinds of TypedDict. TypedDicts are defined to have a property called openness, which can have three states: - Open: all TypedDicts prior to PEP 728 - Closed: no extra keys are allowed (closed=True) - With extra items: extra_items=... from PEP 728 I retained existing text where it made sense but also wrote some from scratch. --- docs/spec/callables.rst | 63 +- docs/spec/glossary.rst | 50 ++ docs/spec/typeddict.rst | 1717 +++++++++++++-------------------------- 3 files changed, 669 insertions(+), 1161 deletions(-) diff --git a/docs/spec/callables.rst b/docs/spec/callables.rst index 56b66bc16..b189c4828 100644 --- a/docs/spec/callables.rst +++ b/docs/spec/callables.rst @@ -180,24 +180,61 @@ generated. For example:: # so **kwargs can contain # a "name" keyword. -Required and non-required keys -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -By default all keys in a ``TypedDict`` are required. This behavior can be -overridden by setting the dictionary's ``total`` parameter as ``False``. -Moreover, :pep:`655` introduced new type qualifiers - ``typing.Required`` and -``typing.NotRequired`` - that enable specifying whether a particular key is -required or not:: - - class Movie(TypedDict): - title: str - year: NotRequired[int] +Required and non-required items +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Items in a TypedDict may be either :term:`required` or :term:`non-required`. When using a ``TypedDict`` to type ``**kwargs`` all of the required and non-required keys should correspond to required and non-required function -keyword parameters. Therefore, if a required key is not supported by the +keyword parameters. Therefore, if a required key is not provided by the caller, then an error must be reported by type checkers. +Read-only items +^^^^^^^^^^^^^^^ + +TypedDict items may also be :term:`read-only`. Marking one or more of the items of a TypedDict +used to type ``**kwargs`` as read-only will have no effect on the type signature of the method. +However, it *will* prevent the item from being modified in the body of the function:: + + class Args(TypedDict): + key1: int + key2: str + + class ReadOnlyArgs(TypedDict): + key1: ReadOnly[int] + key2: ReadOnly[str] + + class Function(Protocol): + def __call__(self, **kwargs: Unpack[Args]) -> None: ... + + def impl(**kwargs: Unpack[ReadOnlyArgs]) -> None: + kwargs["key1"] = 3 # Type check error: key1 is readonly + + fn: Function = impl # Accepted by type checker: function signatures are identical + +Extra items +^^^^^^^^^^^ + +If the TypedDict used for annotating ``**kwargs`` is defined to allow +:term:`extra items`, arbitrary additional keyword arguments of the right +type may be passed to the function:: + + class MovieNoExtra(TypedDict): + name: str + + class MovieExtra(TypedDict, extra_items=int): + name: str + + def f(**kwargs: Unpack[MovieNoExtra]) -> None: ... + def g(**kwargs: Unpack[MovieExtra]) -> None: ... + + # Should be equivalent to: + def f(*, name: str) -> None: ... + def g(*, name: str, **kwargs: int) -> None: ... + + f(name="No Country for Old Men", year=2007) # Not OK. Unrecognized item + g(name="No Country for Old Men", year=2007) # OK + Assignment ^^^^^^^^^^ diff --git a/docs/spec/glossary.rst b/docs/spec/glossary.rst index 1acbcc956..36ef8da45 100644 --- a/docs/spec/glossary.rst +++ b/docs/spec/glossary.rst @@ -27,6 +27,13 @@ This section defines a few terms that may be used elsewhere in the specification ``B``, respectively, such that ``B'`` is a subtype of ``A'``. See :ref:`type-system-concepts`. + closed + A :ref:`TypedDict ` type is closed if it may not contain any + additional :term:`items ` beyond those specified in the TypedDict definition. + A closed TypedDict can be created using the ``closed=True`` argument to + :py:func:`typing.TypedDict`. + Compare :term:`extra items` and :term:`open`. + consistent Two :term:`fully static types ` are "consistent with" each other if they are :term:`equivalent`. Two gradual types are @@ -52,6 +59,14 @@ This section defines a few terms that may be used elsewhere in the specification also materializations of ``B``, and all materializations of ``B`` are also materializations of ``A``. + extra items + A :ref:`TypedDict ` type with extra items may contain arbitrary + additional :term:`items ` beyond those specified in the TypedDict definition, but those + items must be of the type specified by the TypedDict definition. + A TypedDict with extra items can be created using the ``extra_items=`` + argument to :py:func:`typing.TypedDict`. Extra items may or may not be + :term:`read-only`. Compare :term:`closed` and :term:`open`. + fully static type A type is "fully static" if it does not contain any :term:`gradual form`. A fully static type represents a set of possible runtime values. Fully @@ -84,6 +99,12 @@ This section defines a few terms that may be used elsewhere in the specification runtime code using :pep:`526` and :pep:`3107` syntax (the filename ends in ``.py``). + item + In the context of a :ref:`TypedDict `, an item is a key/value + pair defined in the TypedDict definition. Each item has a name (the key) + and a type (the value). Items may be :term:`required` or + :term:`non-required`, and may be :term:`read-only` or writable. + materialize A :term:`gradual type` can be materialized to a more static type (possibly a :term:`fully static type`) by replacing :ref:`Any` with any @@ -110,12 +131,41 @@ This section defines a few terms that may be used elsewhere in the specification ``__class__`` is that type, or any of its subclasses, transitively. In contrast, see :term:`structural` types. + non-required + If an :term:`item` in a :ref:`TypedDict ` is non-required, it may or + may not be present on an object of that TypedDict type, but if it is present + it must be of the type specified by the TypedDict definition. + Items can be marked as non-required using the :py:data:`typing.NotRequired` qualifier + or the ``total=False`` argument to :py:func:`typing.TypedDict`. Compare :term:`required`. + + open + A :ref:`TypedDict ` type is open if it may contain arbitrary + additional :term:`items ` beyond those specified in the TypedDict definition. + This is the default behavior for TypedDicts that do not use the ``closed=True`` + or ``extra_items=`` arguments to :py:func:`typing.TypedDict`. + Open TypedDicts behave similarly to TypedDicts with :term:`extra items` of type + ``ReadOnly[object]``, but differ in some behaviors; see the TypedDict specification + chapter for details. + Compare :term:`extra items` and :term:`closed`. + package A directory or directories that namespace Python modules. (Note the distinction between packages and :term:`distributions `. While most distributions are named after the one package they install, some distributions install multiple packages.) + read-only + A read-only :term:`item` in a :ref:`TypedDict ` may not be modified. + Attempts to assign to or delete that item + should be reported as type errors by a type checker. Read-only items are created + using the :py:data:`typing.ReadOnly` qualifier. + + required + If an :term:`item` in a :ref:`TypedDict ` is required, it must be present + in any object of that TypedDict type. Items are + required by default, but items can also be explicitly marked as required using + the :py:data:`typing.Required` qualifier. Compare :term:`non-required`. + special form A special form is an object that has a special meaning within the type system, comparable to a keyword in the language grammar. Examples include ``Any``, diff --git a/docs/spec/typeddict.rst b/docs/spec/typeddict.rst index 153fd0b76..bb2747457 100644 --- a/docs/spec/typeddict.rst +++ b/docs/spec/typeddict.rst @@ -1,39 +1,54 @@ +.. _`typeddict`: .. _`typed-dictionaries`: Typed dictionaries ================== -.. _`typeddict`: - -TypedDict ---------- - -(Originally specified in :pep:`589`.) - -A TypedDict type represents dictionary objects with a specific set of -string keys, and with specific value types for each valid key. Each -string key can be either required (it must be present) or -non-required (it doesn't need to exist). - -There are two ways of defining TypedDict types. The first uses -a class-based syntax. The second is an alternative -assignment-based syntax that is provided for backwards compatibility, -to allow the feature to be backported to older Python versions. The -rationale is similar to why :pep:`484` supports a comment-based -annotation syntax for Python 2.7: type hinting is particularly useful -for large existing codebases, and these often need to run on older -Python versions. The two syntax options parallel the syntax variants -supported by ``typing.NamedTuple``. Other features include -TypedDict inheritance and totality (specifying whether keys are -required or not). - -This section also provides a sketch of how a type checker is expected to -support type checking operations involving TypedDict objects. Similar to -:pep:`484`, this discussion is left somewhat vague on purpose, to allow -experimentation with a wide variety of different type checking approaches. In -particular, :term:`assignability ` should be :term:`structural`: a -more specific TypedDict type can be assignable to a more general TypedDict -type, without any inheritance relationship between them. +(Originally specified in :pep:`589`, with later additions: ``Required`` +and ``NotRequired`` in :pep:`655`, use with ``Unpack`` in :pep:`692`, +``ReadOnly`` in :pep:`705`, and ``closed=True`` and ``extra_items=`` in :pep:`728`.) + +A TypedDict type represents ``dict`` objects that contain only keys of +type ``str``. There are restrictions on which string keys are valid, and +which values can be associated with each key. Values that are members of a +TypedDict type must be instances of ``dict`` itself, not a subclass. + +TypedDict types can define any number of :term:`items `, which are string +keys associated with values of a specified type. For example, +a TypedDict may contain the item ``a: str``, indicating that the key ``a`` +must map to a value of type ``str``. Items may be either :term:`required`, +meaning they must be present in any instance of the TypedDict type, or +:term:`non-required`, meaning they may be omitted, but if they are present, +they must be of the type specified in the TypedDict definition. By default, +all items in a TypedDict are mutable, but items +may also be marked as :term:`read-only`, indicating that they may not be +modified. + +In addition to explicitly specified items, TypedDicts may allow additional +items. By default, TypedDicts are :term:`open`, meaning they may contain an +unknown set of additional items. They may also be marked as :term:`closed`, +in which case they may not contain any keys beyond those explicitly specified. +As a third option, they may be defined with :term:`extra items` of a specific type. +In this case, there may be any number of additional items present at runtime, but +their values must be of the specified type. Extra items may or may not be +:term:`read-only`. Thus, a TypedDict may be open, closed, or have extra items; +we refer to this property as the *openness* of the TypedDict. For many purposes, +an open TypedDict is equivalent to a TypedDict with read-only extra items of +type ``object``, but certain behaviors differ; for example, the +:ref:`TypedDict constructor ` of open TypedDicts does not +allow unrecognized keys. + +A TypedDict is a :term:`structural` type: independent TypedDict types may be +:term:`assignable` to each other based on their structure, even if they do not +share a common base class. For example, two TypedDict types that contain the same +items are :term:`equivalent`. Nevertheless, TypedDict types may inherit from other +TypedDict types to share common items. TypedDict types may also be generic. + +Syntax +------ + +This section outlines the syntax for creating TypedDict types. There are two +syntaxes: the class-based syntax and the functional syntax. .. _typeddict-class-based-syntax: @@ -41,7 +56,7 @@ Class-based Syntax ^^^^^^^^^^^^^^^^^^ A TypedDict type can be defined using the class definition syntax with -``typing.TypedDict`` as the sole base class:: +``typing.TypedDict`` as a direct or indirect base class:: from typing import TypedDict @@ -52,41 +67,142 @@ A TypedDict type can be defined using the class definition syntax with ``Movie`` is a TypedDict type with two items: ``'name'`` (with type ``str``) and ``'year'`` (with type ``int``). -A type checker should validate that the body of a class-based -TypedDict definition conforms to the following rules: +A TypedDict can also be created through inheritance from one or more +other TypedDict types:: + + class BookBasedMovie(Movie): + based_on: str -* The class body should only contain lines with item definitions of the - form ``key: value_type``, optionally preceded by a docstring. The - syntax for item definitions is identical to attribute annotations, - but there must be no initializer, and the key name actually refers - to the string value of the key instead of an attribute name. +This creates a TypedDict type ``BookBasedMovie`` with three items: +``'name'`` (type ``str``), ``'year'`` (type ``int``), and ``'based_on'`` (type ``str``). +See :ref:`Inheritance ` for more details. -* Type comments cannot be used with the class-based syntax, for - consistency with the class-based ``NamedTuple`` syntax. Instead, - `Alternative Syntax`_ provides an - alternative, assignment-based syntax for backwards compatibility. +A generic TypedDict can be created by inheriting from ``Generic`` with a list +of type parameters:: -* String literal forward references are valid in the value types. + from typing import Generic, TypeVar -* Methods are not allowed, since the runtime type of a TypedDict - object will always be just ``dict`` (it is never a subclass of - ``dict``). + T = TypeVar('T') -* Specifying a metaclass is not allowed. + class Response(TypedDict, Generic[T]): + status: int + payload: T -* TypedDicts may be made generic by adding ``Generic[T]`` among the - bases (or, in Python 3.12 and higher, by using the new - syntax for generic classes). +Or, in Python 3.12 and newer, by using the native syntax for generic classes:: -An empty TypedDict can be created by only including ``pass`` in the -body (if there is a docstring, ``pass`` can be omitted):: + from typing import TypedDict - class EmptyDict(TypedDict): - pass + class Response[T](TypedDict): + status: int + payload: T + +It is invalid to specify a base class other than ``TypedDict``, ``Generic``, +or another TypedDict type in a class-based TypedDict definition. +It is also invalid to specify a custom metaclass. + +A TypedDict definition may also contain the following keyword arguments +in the class definition: + +* ``total``: a boolean literal (``True`` or ``False``) indicating whether + all items are :term:`required` (``True``, the default) or :term:`non-required` + (``False``). This affects only items defined in this class, not in any + base classes, and it does not affect any items that use an explicit + ``Required[]`` or ``NotRequired[]`` qualifier. The value must be exactly + ``True`` or ``False``; other expressions are not allowed. +* ``closed``: a boolean literal (``True`` or ``False``) indicating whether + the TypedDict is :term:`closed` (``True``) or :term:`open` (``False``). + The latter is the default, except when inheriting from another TypedDict that + is not open (see :ref:`typeddict-inheritance`). + As with ``total``, the value must be exactly ``True`` or ``False``. It is an error + to use this argument together with ``extra_items=``. +* ``extra_items``: indicates that the TypedDict has :term:`extra items`. The argument + must be a :term:`annotation expression` specifying the type of the extra items. + The :term:`type qualifier` ``ReadOnly[]`` may be used to indicate that the extra items are + :term:`read-only`. Other type qualifiers are not allowed. If the extra items type + is ``Never``, no extra items are allowed, so this is equivalent to ``closed=True``. + +The body of the class definition defines the :term:`items ` of the TypedDict type. +It may also contain a docstring or ``pass`` statements (primarily to allow the creation of +an empty TypedDict). No other statements are allowed, and type checkers should report an +error if any are present. Type comments are not supported for creating TypedDict items. + +.. _`required-notrequired`: +.. _`required`: +.. _`notrequired`: + +An item definition takes the form of an attribute annotation, ``key: T``. ``key`` is +an identifier and corresponds to the string key of the item, and ``T`` is an +:term:`annotation expression` specifying the type of the item value. This annotation +expression contains a :term:`type expression`, optionally qualified with one of the +:term:`type qualifiers ` ``Required``, ``NotRequired``, or ``ReadOnly``. +These type qualifiers may be nested arbitrarily or wrapped in ``Annotated[]``. It is +an error to use both ``Required`` and ``NotRequired`` in the same item definition. +An item is :term:`read-only` if and only if the ``ReadOnly`` qualifier is used. + +To determine whether an item is :term:`required` or :term:`non-required`, the following +procedure is used: + +* If the ``Required`` qualifier is present, the item is required. +* If the ``NotRequired`` qualifier is present, the item is non-required. +* If the ``total`` argument of the TypedDict definition is ``False``, the item is non-required. +* Else, the item is required. + +It is valid to use ``Required[]`` and ``NotRequired[]`` even for +items where it is redundant, to enable additional explicitness if desired. +Note that the value of ``total`` only affects items defined in the current class body, +not in any base classes. Thus, inheritance can be used to create a TypedDict that mixes +required and non-required items without using ``Required[]`` or ``NotRequired[]``. + +The following example demonstrates some of these rules:: + + from typing import TypedDict, NotRequired, Required, ReadOnly, Annotated + + class Movie(TypedDict): + name: str # required, not read-only + year: int # required, not read-only + director: NotRequired[str] # non-required, not read-only + rating: NotRequired[ReadOnly[float]] # non-required, read-only + invalid: Required[NotRequired[int]] # type checker error: both Required and NotRequired used + + class PartialMovie(TypedDict, total=False): + name: str # non-required, not read-only + year: Required[int] # required, not read-only + score: ReadOnly[float] # non-required, read-only + +.. _typeddict-functional-syntax: + +Functional syntax +^^^^^^^^^^^^^^^^^ + +In addition to the class-based syntax, TypedDict types can be created +using an alternative functional syntax. This syntax allows defining +items with keys that are not valid Python identifiers, and it is compatible +with older Python versions such as 3.5 and 2.7 that don't support the +variable definition syntax introduced in :pep:`526`. On the other hand, this syntax +does not support inheritance. + +The functional syntax resembles the traditional syntax for defining named tuples:: + + from typing import TypedDict + + Movie = TypedDict('Movie', {'name': str, 'year': int}) + +The syntax comprises a call to ``TypedDict()``, the result of which must be immediately +assigned to a variable with the same name as the first argument to ``TypedDict()``. +The call to ``TypedDict()`` must have two positional arguments. The first is a string +literal specifying the name of the TypedDict type. The second is a dictionary specifying +the :term:`items ` of the TypedDict. It must be a dictionary display expression, +not a variable or other expression that evaluates to a dictionary at runtime. +The keys of the dictionary must be string literals and the values must be +:term:`annotation expressions ` following the same rules as +the class-based syntax (i.e., the qualifiers ``Required``, ``NotRequired``, and +``ReadOnly`` are allowed). In addition to the two positional arguments, ``total``, +``closed``, and ``extra_items`` keyword arguments are also supported, with the same +semantics as in the class-based syntax. Using TypedDict Types -^^^^^^^^^^^^^^^^^^^^^ +--------------------- Here is an example of how the type ``Movie`` can be used:: @@ -128,95 +244,232 @@ key, and the ``'name'`` key is missing:: The created TypedDict type object is not a real class object. Here are the only uses of the type a type checker is expected to allow: -* It can be used in type annotations and in any context where an - arbitrary type hint is valid, such as in type aliases and as the - target type of a cast. +* It can be used in :term:`type expressions ` to + represent the TypedDict type. * It can be used as a callable object with keyword arguments - corresponding to the TypedDict items. Non-keyword arguments are not - allowed. Example:: + corresponding to the TypedDict items; see :ref:`typeddict-constructor`. - m = Movie(name='Blade Runner', year=1982) +* It can be used as a base class, but only when defining a derived + TypedDict (see :ref:`above `). - When called, the TypedDict type object returns an ordinary - dictionary object at runtime:: +In particular, TypedDict type objects cannot be used in +``isinstance()`` tests such as ``isinstance(d, Movie)``. This is +consistent with how ``isinstance()`` is not supported for +other type forms such as ``list[str]``. - print(type(m)) # +.. _typeddict-constructor: -* It can be used as a base class, but only when defining a derived - TypedDict. This is discussed in more detail below. +The TypedDict constructor +^^^^^^^^^^^^^^^^^^^^^^^^^ -In particular, TypedDict type objects cannot be used in -``isinstance()`` tests such as ``isinstance(d, Movie)``. The reason is -that there is no existing support for checking types of dictionary -item values, since ``isinstance()`` does not work with many -types, including common ones like ``list[str]``. This would be needed -for cases like this:: +TypedDict types are callable at runtime and can be used as a constructor +to create values that conform to the TypedDict type. The constructor +takes only keyword arguments, corresponding to the items of the TypedDict. +Example:: + + m = Movie(name='Blade Runner', year=1982) + +When called, the TypedDict type object returns an ordinary +dictionary object at runtime:: + + print(type(m)) # + +Every :term:`required` item must be provided as a keyword argument. :term:`Non-required` +items may be omitted. Whether an item is read-only has no effect on the +constructor. + +Closed and open TypedDicts allow no additional items beyond those explicitly +defined, but TypedDicts with extra items allow arbitrary keyword arguments, +which must be of the specified type. Example:: + + from typing import TypedDict, ReadOnly + + class MovieWithExtras(TypedDict, extra_items=ReadOnly[int | str]): + name: str + year: int + + m1 = MovieWithExtras(name='Blade Runner', year=1982) # OK + m2 = MovieWithExtras(name='The Godfather', year=1972, director='Francis Ford Coppola', rating=9) # OK + m3 = MovieWithExtras(name='Inception', year=2010, budget=160.0) # Type check error: budget must be int or str + +Initialization from dictionary literals +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Type checkers should also allow initializing a value of TypedDict type from +a dictionary literal:: - class Strings(TypedDict): - items: list[str] + m: Movie = {'name': 'Blade Runner', 'year': 1982} # OK - print(isinstance({'items': [1]}, Strings)) # Should be False - print(isinstance({'items': ['x']}, Strings)) # Should be True +Or from a call to ``dict()`` with keyword arguments:: -The above use case is not supported. This is consistent with how -``isinstance()`` is not supported for ``list[str]``. + m: Movie = dict(name='Blade Runner', year=1982) # OK + +In these cases, extra keys should not be allowed unless the TypedDict +is defined to allow :term:`extra items`. In this example, the ``director`` key is not defined in +``Movie`` and is expected to generate an error from a type checker:: + + m: Movie = dict( + name='Alien', + year=1979, + director='Ridley Scott') # error: Unexpected key 'director' + +If a TypedDict has extra items, extra keys are allowed, provided their value +matches the extra items type:: + + class ExtraMovie(TypedDict, extra_items=bool): + name: str + + a: ExtraMovie = {"name": "Blade Runner", "novel_adaptation": True} # OK + b: ExtraMovie = { + "name": "Blade Runner", + "year": 1982, # Not OK. 'int' is not assignable to 'bool' + } + +Here, ``extra_items=bool`` specifies that items other than ``'name'`` +have a value type of ``bool`` and are non-required. +.. _typeddict-inheritance: Inheritance -^^^^^^^^^^^ +----------- -It is possible for a TypedDict type to inherit from one or more -TypedDict types using the class-based syntax. In this case the +As discussed under :ref:`typeddict-class-based-syntax`, TypedDict types +can inherit from one or more other TypedDict types. In this case the ``TypedDict`` base class should not be included. Example:: class BookBasedMovie(Movie): based_on: str Now ``BookBasedMovie`` has keys ``name``, ``year``, and ``based_on``. It is -equivalent to this definition, since TypedDict types use :term:`structural` -:term:`assignability `:: +equivalent to this definition, since TypedDict types are :term:`structural` types:: class BookBasedMovie(TypedDict): name: str year: int based_on: str -Here is an example of multiple inheritance:: +Overriding items +^^^^^^^^^^^^^^^^ - class X(TypedDict): - x: int +Under limited circumstances, subclasses may redeclare items defined in a superclass with +a different type or different qualifiers. Redeclaring an item with the same type and qualifiers +is always allowed, although it is redundant. - class Y(TypedDict): - y: str +If an item is mutable in a superclass, it must remain mutable in the subclass. Similarly, +mutable items that are :term:`required` in a superclass must remain required in the subclass, +and mutable :term:`non-required` items in the superclass must remain non-required in the subclass. +However, if the superclass item is :term:`read-only`, a superclass item that is non-required +may be overridden with a required item in the subclass. A read-only item in a superclass +may be redeclared as mutable (that is, without the ``ReadOnly`` qualifier) in a subclass. +These rules are necessary for type safety. - class XYZ(X, Y): - z: bool +If an item is read-only in the superclass, the subclass may redeclare it with a different type +that is :term:`assignable` to the superclass type. Otherwise, changing the type of an item is not allowed. +Example:: -The TypedDict ``XYZ`` has three items: ``x`` (type ``int``), ``y`` -(type ``str``), and ``z`` (type ``bool``). + class X(TypedDict): + x: str + y: ReadOnly[int] -A TypedDict cannot inherit from both a TypedDict type and a -non-TypedDict base class other than ``Generic``. + class Y(X): + x: int # Type check error: cannot overwrite TypedDict field "x" + y: bool # OK: bool is assignable to int, and a mutable item can override a read-only one -Additional notes on TypedDict class inheritance: +Openness +^^^^^^^^ -* Changing a field type of a parent TypedDict class in a subclass is not allowed. - Example:: +The openness of a TypedDict (whether it is :term:`open`, :term:`closed`, or has :term:`extra items`) +is inherited from its superclass by default:: - class X(TypedDict): - x: str + class ClosedBase(TypedDict, closed=True): + name: str - class Y(X): - x: int # Type check error: cannot overwrite TypedDict field "x" + class ClosedChild(ClosedBase): # also closed + pass + + class ExtraItemsBase(TypedDict, extra_items=int | None): + name: str + + class ExtraItemsChild(ExtraItemsBase): # also has extra_items=int | None + pass + +However, subclasses may also explicitly use the ``closed`` and ``extra_items`` arguments +to change the openness of the TypedDict, but in some cases this yields a type checker error. +If the base class is open, all possible states are allowed in the subclass: it may remain open, +it may be closed (with ``closed=True``), or it may have extra items (with ``extra_items=...``). +If the base class is closed, any child classes must also be closed. +If the base class has extra items, but they are not read-only, the child class must also allow +the same extra items. If the base class has read-only extra items, the child class may be closed, +or it may redeclare its extra items with a type that is :term:`assignable` to the base class type. +Child classes may also have mutable extra items if the base class has read-only extra items. + +For example:: + + class ExtraItemsRO(TypedDict, extra_items=ReadOnly[int | str]): + name: str + + class ClosedChild(ExtraItemsRO, closed=True): # OK + pass + + # OK, str is assignable to int | str, and mutable extra items can override read-only ones + class NarrowerChild(ExtraItemsRO, extra_items=str): + pass + +When a TypedDict has extra items, this effectively defines the value type of any unnamed +items accepted to the TypedDict and marks them as non-required. Thus, there are some +restrictions on the items that can be added in subclasses. For each item +added in a subclass of a class with extra items of type ``T``, the following rules must be followed: - In the example outlined above TypedDict class annotations returns - type ``str`` for key ``x``:: +- If ``extra_items`` is read-only + + - The item can be either required or non-required + - The item's value type must be :term:`assignable` to ``T`` + +- If ``extra_items`` is not read-only + + - The item must be non-required + - The item's value type must be :term:`consistent` with ``T`` + +For example:: + + class MovieBase(TypedDict, extra_items=int | None): + name: str + + class MovieRequiredYear(MovieBase): # Not OK. Required key 'year' is not known to 'MovieBase' + year: int | None + + class MovieNotRequiredYear(MovieBase): # Not OK. 'int | None' is not consistent with 'int' + year: NotRequired[int] + + class MovieWithYear(MovieBase): # OK + year: NotRequired[int | None] + + class BookBase(TypedDict, extra_items=ReadOnly[int | str]): + title: str + + class Book(BookBase, extra_items=str): # OK + year: int # OK, since extra_items is read-only + +Multiple inheritance +^^^^^^^^^^^^^^^^^^^^ + +TypedDict types may use multiple inheritance to inherit items from multiple +base classes. Here is an example:: + + class X(TypedDict): + x: int + + class Y(TypedDict): + y: str - print(Y.__annotations__) # {'x': } + class XYZ(X, Y): + z: bool +The TypedDict ``XYZ`` has three items: ``x`` (type ``int``), ``y`` +(type ``str``), and ``z`` (type ``bool``). -* Multiple inheritance does not allow conflict types for the same name field:: +Multiple inheritance does not allow conflicting types for the same item:: class X(TypedDict): x: int @@ -227,195 +480,181 @@ Additional notes on TypedDict class inheritance: class XYZ(X, Y): # Type check error: cannot overwrite TypedDict field "x" while merging xyz: bool +.. _typeddict-assignability: -Totality -^^^^^^^^ - -By default, all keys must be present in a TypedDict. It is possible -to override this by specifying *totality*. Here is how to do this -using the class-based syntax:: - - class Movie(TypedDict, total=False): - name: str - year: int +Subtyping and assignability +--------------------------- -This means that a ``Movie`` TypedDict can have any of the keys omitted. Thus -these are valid:: +Because TypedDict types are :term:`structural` types, a TypedDict ``T1`` is assignable to another +TypedDict type ``T2`` if the two are structurally compatible, meaning that all operations that +are allowed on ``T2`` are also allowed on ``T1``. For similar reasons, TypedDict types are +generally not assignable to any specialization of ``dict`` or ``Mapping``, other than ``Mapping[str, object]``, +though certain :term:`closed` TypedDicts and TypedDicts with :term:`extra items` may be assignable +to these types. - m: Movie = {} - m2: Movie = {'year': 2015} +The rest of this section discusses the :term:`subtyping ` rules for TypedDict in more detail. +As with any type, the rules for :term:`assignability ` can be derived from the subtyping +rules using the :term:`materialization ` procedure. -A type checker is only expected to support a literal ``False`` or -``True`` as the value of the ``total`` argument. ``True`` is the -default, and makes all items defined in the class body be required. +Subtyping between TypedDict types +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The totality flag only applies to items defined in the body of the -TypedDict definition. Inherited items won't be affected, and instead -use totality of the TypedDict type where they were defined. This makes -it possible to have a combination of required and non-required keys in -a single TypedDict type. Alternatively, ``Required`` and ``NotRequired`` -(see below) can be used to mark individual items as required or non-required. +A TypedDict type ``B`` is a :term:`subtype` of a TypedDict type ``A`` if +and only if all of the conditions below are satisfied. For the purposes of these conditions, +an :term:`open` TypedDict is treated as if it had read-only :term:`extra items` of type ``object``. -.. _typeddict-functional-syntax: +The conditions are as follows: -Alternative Syntax -^^^^^^^^^^^^^^^^^^ +- For each item in ``A``: -This section provides an alternative syntax that can be backported to -older Python versions such as 3.5 and 2.7 that don't support the -variable definition syntax introduced in :pep:`526`. It -resembles the traditional syntax for defining named tuples:: + - If it is required in ``A``: - Movie = TypedDict('Movie', {'name': str, 'year': int}) + - It must also be required in ``B``. + - If it is read-only in ``A``, the item type in ``B`` must be a subtype of the item type in ``A``. -It is also possible to specify totality using the alternative syntax:: + - If it is mutable in ``A``, it must also be mutable in ``B``, and the item type in ``B`` must be + :term:`equivalent` to the item type in ``A``. (It follows that for assignability, the two item types + must be :term:`consistent`.) - Movie = TypedDict('Movie', - {'name': str, 'year': int}, - total=False) + - If it is non-required in ``A``: -The semantics are equivalent to the class-based syntax. This syntax -doesn't support inheritance, however. The -motivation for this is keeping the backwards compatible syntax as -simple as possible while covering the most common use cases. + - If it is read-only in ``A``: -A type checker is only expected to accept a dictionary display expression -as the second argument to ``TypedDict``. In particular, a variable that -refers to a dictionary object does not need to be supported, to simplify -implementation. + - If ``B`` has an item with the same key, its item type must be a subtype of the item type in ``A``. + - Else: + - If ``B`` is closed, the check succeeds. + - If ``B`` has extra items, the extra items type must be a subtype of the item type in ``A``. -.. _typeddict-assignability: + - If it is mutable in ``A``: -Assignability -^^^^^^^^^^^^^ + - If ``B`` has an item with the same key, it must also be mutable, and its item type must be + :term:`equivalent` to the item type in ``A``. (As before, it follows that for assignability, the two item types + must be :term:`consistent`.) -First, any TypedDict type is :term:`assignable` to ``Mapping[str, object]``. + - Else: -Second, a TypedDict type ``B`` is :term:`assignable` to a TypedDict ``A`` if -and only if both of these conditions are satisfied: + - If ``B`` is closed, the check fails. + - If ``B`` has extra items, the extra items type must not be read-only and must + be :term:`equivalent` to the item type in ``A``. +- If ``A`` is closed, ``B`` must also be closed, and it must not contain any items that are not present in ``A``. +- If ``A`` has read-only extra items, ``B`` must either be closed or also have extra items, and the extra items type in ``B`` + must be a subtype of the extra items type in ``A``. Additionally, for any items in ``B`` that are not present in ``A``, + the item type must be a subtype of the extra items type in ``A``. +- If ``A`` has mutable extra items, ``B`` must also have mutable extra items, and the extra items type in ``B`` + must be :term:`equivalent` to the extra items type in ``A``. Additionally, for any items in ``B`` that are not present in ``A``, + the item type must be :term:`equivalent` to the extra items type in ``A``. -* For each key in ``A``, ``B`` has the corresponding key and the corresponding - value type in ``B`` is :term:`consistent` with the value type in ``A``. +The intuition behind these rules is that any operation that is valid on ``A`` must also be valid and safe on ``B``. +For example, any key access on ``A`` that is guaranteed to succeed (because the item is required) must also succeed on ``B``, +and any mutating operation (such as setting or deleting a key) that is allowed on ``A`` must also be allowed on ``B``. -* For each required key in ``A``, the corresponding key is required - in ``B``. For each non-required key in ``A``, the corresponding key - is not required in ``B``. +An example where mutability is relevant:: -Discussion: + class A(TypedDict): + x: int | None -* Value types behave invariantly, since TypedDict objects are mutable. - This is similar to mutable container types such as ``List`` and - ``Dict``. Example where this is relevant:: + class B(TypedDict): + x: int - class A(TypedDict): - x: int | None + def f(a: A) -> None: + a['x'] = None - class B(TypedDict): - x: int + b: B = {'x': 0} + f(b) # Type check error: 'B' not assignable to 'A' + b['x'] + 1 # Runtime error: None + 1 - def f(a: A) -> None: - a['x'] = None +.. _typeddict-mapping: - b: B = {'x': 0} - f(b) # Type check error: 'B' not assignable to 'A' - b['x'] + 1 # Runtime error: None + 1 +Subtyping with ``Mapping`` +^^^^^^^^^^^^^^^^^^^^^^^^^^ -* A TypedDict type with a required key is not :term:`assignable` to a TypedDict - type where the same key is a non-required key, since the latter allows keys - to be deleted. Example where this is relevant:: +A TypedDict type is a :term:`subtype` of a type of the form ``Mapping[str, VT]`` +when all value types of the items in the TypedDict +are subtypes of ``VT``. For the purpose of this rule, an :term:`open` TypedDict is considered +to have read-only :term:`extra items` of type ``object``. - class A(TypedDict, total=False): - x: int +For example:: - class B(TypedDict): - x: int + class MovieExtraStr(TypedDict, extra_items=str): + name: str - def f(a: A) -> None: - del a['x'] + extra_str: MovieExtraStr = {"name": "Blade Runner", "summary": ""} + str_mapping: Mapping[str, str] = extra_str # OK - b: B = {'x': 0} - f(b) # Type check error: 'B' not assignable to 'A' - b['x'] + 1 # Runtime KeyError: 'x' + class MovieExtraInt(TypedDict, extra_items=int): + name: str -* A TypedDict type ``A`` with no key ``'x'`` is not :term:`assignable` to a - TypedDict type with a non-required key ``'x'``, since at runtime the key - ``'x'`` could be present and have an :term:`inconsistent ` type - (which may not be visible through ``A`` due to :term:`structural` - assignability). Example:: + extra_int: MovieExtraInt = {"name": "Blade Runner", "year": 1982} + int_mapping: Mapping[str, int] = extra_int # Not OK. 'int | str' is not assignable with 'int' + int_str_mapping: Mapping[str, int | str] = extra_int # OK - class A(TypedDict, total=False): - x: int - y: int +As a consequence, every TypedDict type is :term:`assignable` to ``Mapping[str, object]``. - class B(TypedDict, total=False): - x: int +.. _typeddict-dict: - class C(TypedDict, total=False): - x: int - y: str +Subtyping with ``dict`` +^^^^^^^^^^^^^^^^^^^^^^^ - def f(a: A) -> None: - a['y'] = 1 +Generally, TypedDict types are not subtypes of any specialization of ``dict[...]`` type, since +dictionary types allow destructive operations, including ``clear()``. They +also allow arbitrary keys to be set, which would compromise type safety. - def g(b: B) -> None: - f(b) # Type check error: 'B' not assignable to 'A' +However, a TypedDict with :term:`extra items` may be a subtype of ``dict[str, VT]``, +provided certain conditions are met, because it introduces sufficient restrictions +for this subtyping relation to be safe. +A TypedDict type is a subtype of ``dict[str, VT]`` if the following conditions are met: - c: C = {'x': 0, 'y': 'foo'} - g(c) - c['y'] + 'bar' # Runtime error: int + str +- The TypedDict type has mutable :term:`extra items` of a type that is :term:`equivalent` to ``VT``. +- All items on the TypedDict satisfy the following conditions: + - The value type of the item is :term:`equivalent` to ``VT``. + - The item is not read-only. + - The item is not required. -* A TypedDict isn't :term:`assignable` to any ``Dict[...]`` type, since - dictionary types allow destructive operations, including ``clear()``. They - also allow arbitrary keys to be set, which would compromise type safety. - Example:: +For example:: - class A(TypedDict): - x: int + class IntDict(TypedDict, extra_items=int): + pass - class B(A): - y: str + class IntDictWithNum(IntDict): + num: NotRequired[int] - def f(d: Dict[str, int]) -> None: - d['y'] = 0 + def f(x: IntDict) -> None: + v: dict[str, int] = x # OK + v.clear() # OK - def g(a: A) -> None: - f(a) # Type check error: 'A' not assignable to Dict[str, int] + not_required_num_dict: IntDictWithNum = {"num": 1, "bar": 2} + regular_dict: dict[str, int] = not_required_num_dict # OK + f(not_required_num_dict) # OK - b: B = {'x': 0, 'y': 'foo'} - g(b) - b['y'] + 'bar' # Runtime error: int + str +In this case, methods that are previously unavailable on a TypedDict are allowed, +with signatures matching ``dict[str, VT]`` +(e.g.: ``__setitem__(self, key: str, value: VT) -> None``):: -* A TypedDict with all ``int`` values is not :term:`assignable` to - ``Mapping[str, int]``, since there may be additional non-``int`` values not - visible through the type, due to :term:`structural` assignability. These can - be accessed using the ``values()`` and ``items()`` methods in ``Mapping``, - for example. Example:: + not_required_num_dict.clear() # OK - class A(TypedDict): - x: int + reveal_type(not_required_num_dict.popitem()) # OK. Revealed type is 'tuple[str, int]' - class B(TypedDict): - x: int - y: str + def f(not_required_num_dict: IntDictWithNum, key: str): + not_required_num_dict[key] = 42 # OK + del not_required_num_dict[key] # OK - def sum_values(m: Mapping[str, int]) -> int: - n = 0 - for v in m.values(): - n += v # Runtime error - return n +On the other hand, ``dict[str, VT]`` is not assignable to any TypedDict type, +because such a type includes instances of subclasses of ``dict``:: - def f(a: A) -> None: - sum_values(a) # Error: 'A' not assignable to Mapping[str, int] + class CustomDict(dict[str, int]): + pass - b: B = {'x': 0, 'y': 'foo'} - f(b) + def f(might_not_be_a_builtin_dict: dict[str, int]): + int_dict: IntDict = might_not_be_a_builtin_dict # Not OK + not_a_builtin_dict = CustomDict({"num": 1}) + f(not_a_builtin_dict) .. _typeddict-operations: Supported and Unsupported Operations -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +------------------------------------ Type checkers should support restricted forms of most ``dict`` operations on TypedDict objects. The guiding principle is that @@ -429,64 +668,50 @@ the most important type safety violations to prevent: 3. A key that is not defined in the TypedDict type is added. -A key that is not a literal should generally be rejected, since its -value is unknown during type checking, and thus can cause some of the -above violations. (`Use of Final Values and Literal Types`_ -generalizes this to cover final names and literal types.) - -The use of a key that is not known to exist should be reported as an error, -even if this wouldn't necessarily generate a runtime type error. These are -often mistakes, and these may insert values with an invalid type if -:term:`structural` :term:`assignability ` hides the types of -certain items. For example, ``d['x'] = 1`` should generate a type check error -if ``'x'`` is not a valid key for ``d`` (which is assumed to be a TypedDict -type). - -Extra keys included in TypedDict object construction should also be -caught. In this example, the ``director`` key is not defined in -``Movie`` and is expected to generate an error from a type checker:: - - m: Movie = dict( - name='Alien', - year=1979, - director='Ridley Scott') # error: Unexpected key 'director' +4. Read-only items are modified or deleted. -Type checkers should reject the following operations on TypedDict -objects as unsafe, even though they are valid for normal dictionaries: +.. _`readonly`: -* Operations with arbitrary ``str`` keys (instead of string literals - or other expressions with known string values) should generally be - rejected. This involves both destructive operations such as setting - an item and read-only operations such as subscription expressions. - As an exception to the above rule, ``d.get(e)`` and ``e in d`` - should be allowed for TypedDict objects, for an arbitrary expression - ``e`` with type ``str``. The motivation is that these are safe and - can be useful for introspecting TypedDict objects. The static type - of ``d.get(e)`` should be ``object`` if the string value of ``e`` - cannot be determined statically. +Items that are :term:`read-only` may not be mutated (added, modified, or removed):: -* ``clear()`` is not safe since it could remove required keys, some of which - may not be directly visible because of :term:`structural` - :term:`assignability `. ``popitem()`` is similarly unsafe, even - if all known keys are not required (``total=False``). + from typing import ReadOnly -* ``del obj['key']`` should be rejected unless ``'key'`` is a - non-required key. + class Band(TypedDict): + name: str + members: ReadOnly[list[str]] -Type checkers may allow reading an item using ``d['x']`` even if -the key ``'x'`` is not required, instead of requiring the use of -``d.get('x')`` or an explicit ``'x' in d`` check. The rationale is -that tracking the existence of keys is difficult to implement in full -generality, and that disallowing this could require many changes to -existing code. + blur: Band = {"name": "blur", "members": []} + blur["name"] = "Blur" # OK: "name" is not read-only + blur["members"] = ["Damon Albarn"] # Type check error: "members" is read-only + blur["members"].append("Damon Albarn") # OK: list is mutable The exact type checking rules are up to each type checker to decide. In some cases potentially unsafe operations may be accepted if the alternative is to generate false positive errors for idiomatic code. +Sometimes, operations on :term:`closed` TypedDicts or TypedDicts with +:term:`extra items` are safe even if they would be unsafe on +:term:`open` TypedDicts, so type checker behavior may depend on the +openness of the TypedDict. + +Allowed keys +^^^^^^^^^^^^ + +Many operations on TypedDict objects involve specifying a dictionary key. +Examples include accessing an item with ``d['key']`` or setting an item with +``d['key'] = value``. +A key that is not a literal should generally be rejected, since its +value is unknown during type checking, and thus can cause some of the +above violations. This involves both destructive operations such as setting +an item and read-only operations such as subscription expressions. -Use of Final Values and Literal Types -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The use of a key that is not known to exist should be reported as an error, +even if this wouldn't necessarily generate a runtime type error. These are +often mistakes, and these may insert values with an invalid type if +:term:`structural` :term:`assignability ` hides the types of +certain items. For example, ``d['x'] = 1`` should generate a type check error +if ``'x'`` is not a valid key for ``d`` (which is assumed to be a TypedDict +type), unless ``d`` has mutable :term:`extra items` of a compatible type. Type checkers should allow :ref:`final names ` with string values to be used instead of string literals in operations on @@ -504,307 +729,63 @@ can be used instead of a literal value:: key: Literal['year', 'name']) -> int | str: return movie[key] -Type checkers are only expected to support actual string literals, not -final names or literal types, for specifying keys in a TypedDict type -definition. Also, only a boolean literal can be used to specify -totality in a TypedDict definition. The motivation for this is to -make type declarations self-contained, and to simplify the -implementation of type checkers. - - -Backwards Compatibility -^^^^^^^^^^^^^^^^^^^^^^^ +Specific operations +^^^^^^^^^^^^^^^^^^^ -To retain backwards compatibility, type checkers should not infer a -TypedDict type unless it is sufficiently clear that this is desired by -the programmer. When unsure, an ordinary dictionary type should be -inferred. Otherwise existing code that type checks without errors may -start generating errors once TypedDict support is added to the type -checker, since TypedDict types are more restrictive than dictionary -types. In particular, they aren't subtypes of dictionary types. +This section discusses some specific operations in more detail. -.. _`required-notrequired`: +* As an exception to the general rule around non-literal keys, ``d.get(e)`` and ``e in d`` + should be allowed for TypedDict objects, for an arbitrary expression + ``e`` with type ``str``. The motivation is that these are safe and + can be useful for introspecting TypedDict objects. The static type + of ``d.get(e)`` should be the union of all possible item types in ``d`` + if the string value of ``e`` cannot be determined statically. + (This simplifies to ``object`` if ``d`` is :term:`open`.) -``Required`` and ``NotRequired`` --------------------------------- +* ``clear()`` is not safe on :term:`open` TypedDicts since it could remove required keys, some of which + may not be directly visible because of :term:`structural` + :term:`assignability `. However, this method is safe on + :term:`closed` TypedDicts and TypedDicts with :term:`extra items` if + there are no required or read-only items and there cannot be any subclasses with required + or read-only items. -(Originally specified in :pep:`655`.) +* ``popitem()`` is similarly unsafe on many TypedDicts, even + if all known keys are not required (``total=False``). -.. _`required`: +* ``del obj['key']`` should be rejected unless ``'key'`` is a + non-required, mutable key. + +* Type checkers may allow reading an item using ``d['x']`` even if + the key ``'x'`` is not required, instead of requiring the use of + ``d.get('x')`` or an explicit ``'x' in d`` check. The rationale is + that tracking the existence of keys is difficult to implement in full + generality, and that disallowing this could require many changes to + existing code. + Similarly, type checkers may allow indexed accesses + with arbitrary str keys when a TypedDict is :term:`closed` or has :term:`extra items`. + For example:: -The ``typing.Required`` :term:`type qualifier` is used to indicate that a -variable declared in a TypedDict definition is a required key: + def bar(movie: MovieExtraInt, key: str) -> None: + reveal_type(movie[key]) # Revealed type is 'str | int' -:: +* The return types of the ``items()`` and ``values()`` methods can be determined + from the union of all item types in the TypedDict (which would include ``object`` + for :term:`open` TypedDicts). Therefore, type checkers should infer more precise + types for TypedDicts that are not open:: - class Movie(TypedDict, total=False): - title: Required[str] - year: int + from typing import TypedDict -.. _`notrequired`: + class MovieExtraInt(TypedDict, extra_items=int): + name: str -Additionally the ``typing.NotRequired`` :term:`type qualifier` is used to -indicate that a variable declared in a TypedDict definition is a -potentially-missing key: + def foo(movie: MovieExtraInt) -> None: + reveal_type(movie.items()) # Revealed type is 'dict_items[str, str | int]' + reveal_type(movie.values()) # Revealed type is 'dict_values[str, str | int]' -:: - - class Movie(TypedDict): # implicitly total=True - title: str - year: NotRequired[int] - -It is an error to use ``Required[]`` or ``NotRequired[]`` in any -location that is not an item of a TypedDict. -Type checkers must enforce this restriction. - -It is valid to use ``Required[]`` and ``NotRequired[]`` even for -items where it is redundant, to enable additional explicitness if desired: - -:: - - class Movie(TypedDict): - title: Required[str] # redundant - year: NotRequired[int] - -It is an error to use both ``Required[]`` and ``NotRequired[]`` at the -same time: - -:: - - class Movie(TypedDict): - title: str - year: NotRequired[Required[int]] # ERROR - -Type checkers must enforce this restriction. -The runtime implementations of ``Required[]`` and ``NotRequired[]`` -may also enforce this restriction. - -The :ref:`alternative functional syntax ` -for TypedDict also supports -``Required[]``, ``NotRequired[]``, and ``ReadOnly[]``: - -:: - - Movie = TypedDict('Movie', {'name': str, 'year': NotRequired[int]}) - - -Interaction with ``total=False`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Any TypedDict declared with ``total=False`` is equivalent -to a TypedDict with an implicit ``total=True`` definition with all of its -keys marked as ``NotRequired[]``. - -Therefore: - -:: - - class _MovieBase(TypedDict): # implicitly total=True - title: str - - class Movie(_MovieBase, total=False): - year: int - - -is equivalent to: - -:: - - class _MovieBase(TypedDict): - title: str - - class Movie(_MovieBase): - year: NotRequired[int] - - -Interaction with ``Annotated[]`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -``Required[]`` and ``NotRequired[]`` can be used with ``Annotated[]``, -in any nesting order: - -:: - - class Movie(TypedDict): - title: str - year: NotRequired[Annotated[int, ValueRange(-9999, 9999)]] # ok - -:: - - class Movie(TypedDict): - title: str - year: Annotated[NotRequired[int], ValueRange(-9999, 9999)] # ok - -In particular allowing ``Annotated[]`` to be the outermost annotation -for an item allows better interoperability with non-typing uses of -annotations, which may always want ``Annotated[]`` as the outermost annotation -(`discussion `__). - - -Read-only Items ---------------- - -(Originally specified in :pep:`705`.) - -.. _`readonly`: - -``typing.ReadOnly`` type qualifier -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The ``typing.ReadOnly`` :term:`type qualifier` is used to indicate that an item declared in a ``TypedDict`` definition may not be mutated (added, modified, or removed):: - - from typing import ReadOnly - - class Band(TypedDict): - name: str - members: ReadOnly[list[str]] - - blur: Band = {"name": "blur", "members": []} - blur["name"] = "Blur" # OK: "name" is not read-only - blur["members"] = ["Damon Albarn"] # Type check error: "members" is read-only - blur["members"].append("Damon Albarn") # OK: list is mutable - - -Interaction with other special types -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -``ReadOnly[]`` can be used with ``Required[]``, ``NotRequired[]`` and ``Annotated[]``, in any nesting order: - -:: - - class Movie(TypedDict): - title: ReadOnly[Required[str]] # OK - year: ReadOnly[NotRequired[Annotated[int, ValueRange(-9999, 9999)]]] # OK - -:: - - class Movie(TypedDict): - title: Required[ReadOnly[str]] # OK - year: Annotated[NotRequired[ReadOnly[int]], ValueRange(-9999, 9999)] # OK - - -Inheritance -^^^^^^^^^^^ - -Subclasses can redeclare read-only items as non-read-only, allowing them to be mutated:: - - class NamedDict(TypedDict): - name: ReadOnly[str] - - class Album(NamedDict): - name: str - year: int - - album: Album = { "name": "Flood", "year": 1990 } - album["year"] = 1973 - album["name"] = "Dark Side Of The Moon" # OK: "name" is not read-only in Album - -If a read-only item is not redeclared, it remains read-only:: - - class Album(NamedDict): - year: int - - album: Album = { "name": "Flood", "year": 1990 } - album["name"] = "Dark Side Of The Moon" # Type check error: "name" is read-only in Album - -Subclasses can narrow value types of read-only items:: - - class AlbumCollection(TypedDict): - albums: ReadOnly[Collection[Album]] - - class RecordShop(AlbumCollection): - name: str - albums: ReadOnly[list[Album]] # OK: "albums" is read-only in AlbumCollection - -Subclasses can require items that are read-only but not required in the superclass:: - - class OptionalName(TypedDict): - name: ReadOnly[NotRequired[str]] - - class RequiredName(OptionalName): - name: ReadOnly[Required[str]] - - d: RequiredName = {} # Type check error: "name" required - -Subclasses can combine these rules:: - - class OptionalIdent(TypedDict): - ident: ReadOnly[NotRequired[str | int]] - - class User(OptionalIdent): - ident: str # Required, mutable, and not an int - -Note that these are just consequences of :term:`structural` typing, but they -are highlighted here as the behavior now differs from the rules specified in -:pep:`589`. - -Assignability -^^^^^^^^^^^^^ - -*This section updates the assignability rules described above that were created -prior to the introduction of ReadOnly* - -A TypedDict type ``B`` is :term:`assignable` to a TypedDict type ``A`` if ``B`` -is :term:`structurally ` assignable to ``A``. This is true if and -only if all of the following are satisfied: - -* For each item in ``A``, ``B`` has the corresponding key, unless the item in - ``A`` is read-only, not required, and of top value type - (``ReadOnly[NotRequired[object]]``). -* For each item in ``A``, if ``B`` has the corresponding key, the corresponding - value type in ``B`` is assignable to the value type in ``A``. -* For each non-read-only item in ``A``, its value type is assignable to the - corresponding value type in ``B``, and the corresponding key is not read-only - in ``B``. -* For each required key in ``A``, the corresponding key is required in ``B``. -* For each non-required key in ``A``, if the item is not read-only in ``A``, - the corresponding key is not required in ``B``. - -Discussion: - -* All non-specified items in a TypedDict implicitly have value type - ``ReadOnly[NotRequired[object]]``. - -* Read-only items behave covariantly, as they cannot be mutated. This is - similar to container types such as ``Sequence``, and different from - non-read-only items, which behave invariantly. Example:: - - class A(TypedDict): - x: ReadOnly[int | None] - - class B(TypedDict): - x: int - - def f(a: A) -> None: - print(a["x"] or 0) - - b: B = {"x": 1} - f(b) # Accepted by type checker - -* A TypedDict type ``A`` with no explicit key ``'x'`` is not :term:`assignable` - to a TypedDict type ``B`` with a non-required key ``'x'``, since at runtime - the key ``'x'`` could be present and have an :term:`inconsistent - ` type (which may not be visible through ``A`` due to - :term:`structural` typing). The only exception to this rule is if the item in - ``B`` is read-only, and the value type is of top type (``object``). For - example:: - - class A(TypedDict): - x: int - - class B(TypedDict): - x: int - y: ReadOnly[NotRequired[object]] - - a: A = { "x": 1 } - b: B = a # Accepted by type checker - -Update method -^^^^^^^^^^^^^ - -In addition to existing type checking rules, type checkers should error if a -TypedDict with a read-only item is updated with another TypedDict that declares -that key:: +* The ``update()`` method should not allow mutating a read-only item. + Therefore, type checkers should error if a + TypedDict with a read-only item is updated with another TypedDict that declares + that key:: class A(TypedDict): x: ReadOnly[int] @@ -814,7 +795,7 @@ that key:: a2: A = { "x": 3, "y": 4 } a1.update(a2) # Type check error: "x" is read-only in A -Unless the declared value is of bottom type (:data:`~typing.Never`):: + Unless the declared value is of bottom type (:data:`~typing.Never`):: class B(TypedDict): x: NotRequired[typing.Never] @@ -823,575 +804,15 @@ Unless the declared value is of bottom type (:data:`~typing.Never`):: def update_a(a: A, b: B) -> None: a.update(b) # Accepted by type checker: "x" cannot be set on b -Note: Nothing will ever match the ``Never`` type, so an item annotated with it must be absent. - -Keyword argument typing -^^^^^^^^^^^^^^^^^^^^^^^ - -As discussed in the section :ref:`unpack-kwargs`, an unpacked ``TypedDict`` can be used to annotate ``**kwargs``. Marking one or more of the items of a ``TypedDict`` used in this way as read-only will have no effect on the type signature of the method. However, it *will* prevent the item from being modified in the body of the function:: - - class Args(TypedDict): - key1: int - key2: str - - class ReadOnlyArgs(TypedDict): - key1: ReadOnly[int] - key2: ReadOnly[str] - - class Function(Protocol): - def __call__(self, **kwargs: Unpack[Args]) -> None: ... - - def impl(**kwargs: Unpack[ReadOnlyArgs]) -> None: - kwargs["key1"] = 3 # Type check error: key1 is readonly - - fn: Function = impl # Accepted by type checker: function signatures are identical - -Extra Items and Closed TypedDicts -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -(Originally specified in :pep:`728`.) - -This section discusses the ``extra_items`` and ``closed`` class parameters. - -If ``extra_items`` is specified, extra items are treated as :ref:`non-required -` -items matching the ``extra_items`` argument, whose keys are allowed when -determining :ref:`supported and unsupported operations -`. - -The ``extra_items`` Class Parameter ------------------------------------ - -By default ``extra_items`` is unset. For a TypedDict type that specifies -``extra_items``, during construction, the value type of each unknown item -is expected to be non-required and assignable to the ``extra_items`` argument. -For example:: - - class Movie(TypedDict, extra_items=bool): - name: str - - a: Movie = {"name": "Blade Runner", "novel_adaptation": True} # OK - b: Movie = { - "name": "Blade Runner", - "year": 1982, # Not OK. 'int' is not assignable to 'bool' - } - -Here, ``extra_items=bool`` specifies that items other than ``'name'`` -have a value type of ``bool`` and are non-required. - -The alternative inline syntax is also supported:: - - Movie = TypedDict("Movie", {"name": str}, extra_items=bool) - -Accessing extra items is allowed. Type checkers must infer their value type from -the ``extra_items`` argument:: - - def f(movie: Movie) -> None: - reveal_type(movie["name"]) # Revealed type is 'str' - reveal_type(movie["novel_adaptation"]) # Revealed type is 'bool' - -``extra_items`` is inherited through subclassing:: - - class MovieBase(TypedDict, extra_items=ReadOnly[int | None]): - name: str - - class Movie(MovieBase): - year: int - - a: Movie = {"name": "Blade Runner", "year": None} # Not OK. 'None' is incompatible with 'int' - b: Movie = { - "name": "Blade Runner", - "year": 1982, - "other_extra_key": None, - } # OK - -Here, ``'year'`` in ``a`` is an extra key defined on ``Movie`` whose value type -is ``int``. ``'other_extra_key'`` in ``b`` is another extra key whose value type -must be assignable to the value of ``extra_items`` defined on ``MovieBase``. - -.. _typed-dict-closed: - -The ``closed`` Class Parameter ------------------------------- - -When neither ``extra_items`` nor ``closed=True`` is specified, ``closed=False`` -is assumed. The TypedDict should allow non-required extra items of value type -``ReadOnly[object]`` during inheritance or assignability checks, to -preserve the default TypedDict behavior. Extra keys included in TypedDict -object construction should still be caught, as mentioned :ref:`above `. - -When ``closed=True`` is set, no extra items are allowed. This is equivalent to -``extra_items=Never``, because there can't be a value type that is assignable to -:class:`~typing.Never`. It is a runtime error to use the ``closed`` and -``extra_items`` parameters in the same TypedDict definition. - -Similar to ``total``, only a literal ``True`` or ``False`` is supported as the -value of the ``closed`` argument. Type checkers should reject any non-literal value. - -Passing ``closed=False`` explicitly requests the default TypedDict behavior, -where arbitrary other keys may be present and subclasses may add arbitrary items. -It is a type checker error to pass ``closed=False`` if a superclass has -``closed=True`` or sets ``extra_items``. - -If ``closed`` is not provided, the behavior is inherited from the superclass. -If the superclass is TypedDict itself or the superclass does not have ``closed=True`` -or the ``extra_items`` parameter, the previous TypedDict behavior is preserved: -arbitrary extra items are allowed. If the superclass has ``closed=True``, the -child class is also closed:: - - class BaseMovie(TypedDict, closed=True): - name: str - - class MovieA(BaseMovie): # OK, still closed - pass - - class MovieB(BaseMovie, closed=True): # OK, but redundant - pass - - class MovieC(BaseMovie, closed=False): # Type checker error - pass - -As a consequence of ``closed=True`` being equivalent to ``extra_items=Never``, -the same rules that apply to ``extra_items=Never`` also apply to -``closed=True``. While they both have the same effect, ``closed=True`` is -preferred over ``extra_items=Never``. - -It is possible to use ``closed=True`` when subclassing if the ``extra_items`` -argument is a read-only type:: - - class Movie(TypedDict, extra_items=ReadOnly[str]): - pass - - class MovieClosed(Movie, closed=True): # OK - pass - - class MovieNever(Movie, extra_items=Never): # OK, but 'closed=True' is preferred - pass - -This will be further discussed in -:ref:`a later section `. - -``closed`` is also supported with the functional syntax:: - - Movie = TypedDict("Movie", {"name": str}, closed=True) - -Interaction with Totality -------------------------- - -It is an error to use ``Required[]`` or ``NotRequired[]`` with ``extra_items``. -``total=False`` and ``total=True`` have no effect on ``extra_items`` itself. - -The extra items are non-required, regardless of the `totality -`__ of the -TypedDict. :ref:`Operations ` -that are available to ``NotRequired`` items should also be available to the -extra items:: - - class Movie(TypedDict, extra_items=int): - name: str - - def f(movie: Movie) -> None: - del movie["name"] # Not OK. The value type of 'name' is 'Required[int]' - del movie["year"] # OK. The value type of 'year' is 'NotRequired[int]' - -Interaction with ``Unpack`` ---------------------------- - -For type checking purposes, ``Unpack[SomeTypedDict]`` with extra items should be -treated as its equivalent in regular parameters, and the existing rules for -function parameters still apply:: - - class MovieNoExtra(TypedDict): - name: str - - class MovieExtra(TypedDict, extra_items=int): - name: str - - def f(**kwargs: Unpack[MovieNoExtra]) -> None: ... - def g(**kwargs: Unpack[MovieExtra]) -> None: ... - - # Should be equivalent to: - def f(*, name: str) -> None: ... - def g(*, name: str, **kwargs: int) -> None: ... - - f(name="No Country for Old Men", year=2007) # Not OK. Unrecognized item - g(name="No Country for Old Men", year=2007) # OK - -Interaction with Read-only Items --------------------------------- - -When the ``extra_items`` argument is annotated with the ``ReadOnly[]`` -:term:`type qualifier`, the extra items on the TypedDict have the -properties of read-only items. This interacts with inheritance rules specified -in :ref:`Read-only Items `. - -Notably, if the TypedDict type specifies ``extra_items`` to be read-only, -subclasses of the TypedDict type may redeclare ``extra_items``. - -Because a non-closed TypedDict type implicitly allows non-required extra items -of value type ``ReadOnly[object]``, its subclass can override the -``extra_items`` argument with more specific types. - -More details are discussed in the later sections. - -Inheritance ------------ - -``extra_items`` is inherited in a similar way as a regular ``key: value_type`` -item. As with the other keys, the `inheritance rules -`__ -and :ref:`Read-only Items ` inheritance rules apply. - -We need to reinterpret these rules to define how ``extra_items`` interacts with -them. - - * Changing a field type of a parent TypedDict class in a subclass is not allowed. - -First, it is not allowed to change the value of ``extra_items`` in a subclass -unless it is declared to be ``ReadOnly`` in the superclass:: - - class Parent(TypedDict, extra_items=int | None): - pass - - class Child(Parent, extra_items=int): # Not OK. Like any other TypedDict item, extra_items's type cannot be changed - pass - -Second, ``extra_items=T`` effectively defines the value type of any unnamed -items accepted to the TypedDict and marks them as non-required. Thus, the above -restriction applies to any additional items defined in a subclass. For each item -added in a subclass, all of the following conditions should apply: - -.. _pep728-inheritance-read-only: - -- If ``extra_items`` is read-only - - - The item can be either required or non-required - - - The item's value type is :term:`assignable` to ``T`` - -- If ``extra_items`` is not read-only - - - The item is non-required - - - The item's value type is :term:`consistent` with ``T`` - -- If ``extra_items`` is not overridden, the subclass inherits it as-is. - -For example:: - - class MovieBase(TypedDict, extra_items=int | None): - name: str - - class MovieRequiredYear(MovieBase): # Not OK. Required key 'year' is not known to 'MovieBase' - year: int | None - - class MovieNotRequiredYear(MovieBase): # Not OK. 'int | None' is not consistent with 'int' - year: NotRequired[int] - - class MovieWithYear(MovieBase): # OK - year: NotRequired[int | None] - - class BookBase(TypedDict, extra_items=ReadOnly[int | str]): - title: str - - class Book(BookBase, extra_items=str): # OK - year: int # OK - -An important side effect of the inheritance rules is that we can define a -TypedDict type that disallows additional items:: - - class MovieClosed(TypedDict, extra_items=Never): - name: str - -Here, passing the value :class:`~typing.Never` to ``extra_items`` specifies that -there can be no other keys in ``MovieFinal`` other than the known ones. -Because of its potential common use, there is a preferred alternative:: - - class MovieClosed(TypedDict, closed=True): - name: str - -where we implicitly assume that ``extra_items=Never``. - -Assignability -------------- - -Let ``S`` be the set of keys of the explicitly defined items on a TypedDict -type. If it specifies ``extra_items=T``, the TypedDict type is considered to -have an infinite set of items that all satisfy the following conditions. - -- If ``extra_items`` is read-only: - - - The key's value type is :term:`assignable` to ``T``. - - - The key is not in ``S``. - -- If ``extra_items`` is not read-only: - - - The key is non-required. - - - The key's value type is :term:`consistent` with ``T``. - - - The key is not in ``S``. - -For type checking purposes, let ``extra_items`` be a non-required pseudo-item -when checking for assignability according to rules defined in the -:ref:`Read-only Items ` section, with a new rule added in bold -text as follows: - - A TypedDict type ``B`` is :term:`assignable` to a TypedDict type - ``A`` if ``B`` is :term:`structurally ` assignable to - ``A``. This is true if and only if all of the following are satisfied: - - * **[If no key with the same name can be found in ``B``, the 'extra_items' - argument is considered the value type of the corresponding key.]** - - * For each item in ``A``, ``B`` has the corresponding key, unless the item in - ``A`` is read-only, not required, and of top value type - (``ReadOnly[NotRequired[object]]``). - - * For each item in ``A``, if ``B`` has the corresponding key, the corresponding - value type in ``B`` is assignable to the value type in ``A``. - - * For each non-read-only item in ``A``, its value type is assignable to the - corresponding value type in ``B``, and the corresponding key is not read-only - in ``B``. - - * For each required key in ``A``, the corresponding key is required in ``B``. - - * For each non-required key in ``A``, if the item is not read-only in ``A``, - the corresponding key is not required in ``B``. - -The following examples illustrate these checks in action. - -``extra_items`` puts various restrictions on additional items for assignability -checks:: - - class Movie(TypedDict, extra_items=int | None): - name: str - - class MovieDetails(TypedDict, extra_items=int | None): - name: str - year: NotRequired[int] - - details: MovieDetails = {"name": "Kill Bill Vol. 1", "year": 2003} - movie: Movie = details # Not OK. While 'int' is assignable to 'int | None', - # 'int | None' is not assignable to 'int' - - class MovieWithYear(TypedDict, extra_items=int | None): - name: str - year: int | None - - details: MovieWithYear = {"name": "Kill Bill Vol. 1", "year": 2003} - movie: Movie = details # Not OK. 'year' is not required in 'Movie', - # but it is required in 'MovieWithYear' - -where ``MovieWithYear`` (B) is not assignable to ``Movie`` (A) -according to this rule: - - * For each non-required key in ``A``, if the item is not read-only in ``A``, - the corresponding key is not required in ``B``. - -When ``extra_items`` is specified to be read-only on a TypedDict type, it is -possible for an item to have a :term:`narrower ` type than the -``extra_items`` argument:: - - class Movie(TypedDict, extra_items=ReadOnly[str | int]): - name: str - - class MovieDetails(TypedDict, extra_items=int): - name: str - year: NotRequired[int] - - details: MovieDetails = {"name": "Kill Bill Vol. 2", "year": 2004} - movie: Movie = details # OK. 'int' is assignable to 'str | int'. - -This behaves the same way as if ``year: ReadOnly[str | int]`` is an item -explicitly defined in ``Movie``. - -``extra_items`` as a pseudo-item follows the same rules that other items have, -so when both TypedDicts types specify ``extra_items``, this check is naturally -enforced:: - - class MovieExtraInt(TypedDict, extra_items=int): - name: str - - class MovieExtraStr(TypedDict, extra_items=str): - name: str - - extra_int: MovieExtraInt = {"name": "No Country for Old Men", "year": 2007} - extra_str: MovieExtraStr = {"name": "No Country for Old Men", "description": ""} - extra_int = extra_str # Not OK. 'str' is not assignable to extra items type 'int' - extra_str = extra_int # Not OK. 'int' is not assignable to extra items type 'str' - -A non-closed TypedDict type implicitly allows non-required extra keys of value -type ``ReadOnly[object]``. Applying the assignability rules between this type -and a closed TypedDict type is allowed:: - - class MovieNotClosed(TypedDict): - name: str - - extra_int: MovieExtraInt = {"name": "No Country for Old Men", "year": 2007} - not_closed: MovieNotClosed = {"name": "No Country for Old Men"} - extra_int = not_closed # Not OK. - # 'extra_items=ReadOnly[object]' implicitly on 'MovieNotClosed' - # is not assignable to with 'extra_items=int' - not_closed = extra_int # OK - -Interaction with Constructors ------------------------------ - -TypedDicts that allow extra items of type ``T`` also allow arbitrary keyword -arguments of this type when constructed by calling the class object:: + Note: Nothing will ever match the ``Never`` type, so an item annotated with it must be absent. - class NonClosedMovie(TypedDict): - name: str - - NonClosedMovie(name="No Country for Old Men") # OK - NonClosedMovie(name="No Country for Old Men", year=2007) # Not OK. Unrecognized item - - class ExtraMovie(TypedDict, extra_items=int): - name: str - - ExtraMovie(name="No Country for Old Men") # OK - ExtraMovie(name="No Country for Old Men", year=2007) # OK - ExtraMovie( - name="No Country for Old Men", - language="English", - ) # Not OK. Wrong type for extra item 'language' - - # This implies 'extra_items=Never', - # so extra keyword arguments would produce an error - class ClosedMovie(TypedDict, closed=True): - name: str - - ClosedMovie(name="No Country for Old Men") # OK - ClosedMovie( - name="No Country for Old Men", - year=2007, - ) # Not OK. Extra items not allowed - -Supported and Unsupported Operations ------------------------------------- - -This statement from :ref:`above ` still holds true. - - Operations with arbitrary str keys (instead of string literals or other - expressions with known string values) should generally be rejected. - -Operations that already apply to ``NotRequired`` items should generally also -apply to extra items, following the same rationale from :ref:`above `: - - The exact type checking rules are up to each type checker to decide. In some - cases potentially unsafe operations may be accepted if the alternative is to - generate false positive errors for idiomatic code. - -Some operations, including indexed accesses and assignments with arbitrary str keys, -may be allowed due to the TypedDict being :term:`assignable` to -``Mapping[str, VT]`` or ``dict[str, VT]``. The two following sections will expand -on that. - -Interaction with Mapping[str, VT] ---------------------------------- - -A TypedDict type is :term:`assignable` to a type of the form ``Mapping[str, VT]`` -when all value types of the items in the TypedDict -are assignable to ``VT``. For the purpose of this rule, a -TypedDict that does not have ``extra_items=`` or ``closed=`` set is considered -to have an item with a value of type ``ReadOnly[object]``. This extends the -general rule for :ref:`TypedDict assignability `. - -For example:: - - class MovieExtraStr(TypedDict, extra_items=str): - name: str - - extra_str: MovieExtraStr = {"name": "Blade Runner", "summary": ""} - str_mapping: Mapping[str, str] = extra_str # OK - - class MovieExtraInt(TypedDict, extra_items=int): - name: str - - extra_int: MovieExtraInt = {"name": "Blade Runner", "year": 1982} - int_mapping: Mapping[str, int] = extra_int # Not OK. 'int | str' is not assignable with 'int' - int_str_mapping: Mapping[str, int | str] = extra_int # OK - -Type checkers should infer the precise signatures of ``values()`` and ``items()`` -on such TypedDict types:: - - def foo(movie: MovieExtraInt) -> None: - reveal_type(movie.items()) # Revealed type is 'dict_items[str, str | int]' - reveal_type(movie.values()) # Revealed type is 'dict_values[str, str | int]' - -By extension of this assignability rule, type checkers may allow indexed accesses -with arbitrary str keys when ``extra_items`` or ``closed=True`` is specified. -For example:: - - def bar(movie: MovieExtraInt, key: str) -> None: - reveal_type(movie[key]) # Revealed type is 'str | int' - -.. _pep728-type-narrowing: - -Defining the type narrowing behavior for TypedDict is out-of-scope for this spec. -This leaves flexibility for a type checker to be more/less restrictive about -indexed accesses with arbitrary str keys. For example, a type checker may opt -for more restriction by requiring an explicit ``'x' in d`` check. - -Interaction with dict[str, VT] ------------------------------- - -Because the presence of ``extra_items`` on a closed TypedDict type -prohibits additional required keys in its :term:`structural` -:term:`subtypes `, we can determine if the TypedDict type and -its structural subtypes will ever have any required key during static analysis. - -The TypedDict type is :term:`assignable` to ``dict[str, VT]`` if all -items on the TypedDict type satisfy the following conditions: - -- The value type of the item is :term:`consistent` with ``VT``. - -- The item is not read-only. - -- The item is not required. - -For example:: - - class IntDict(TypedDict, extra_items=int): - pass - - class IntDictWithNum(IntDict): - num: NotRequired[int] - - def f(x: IntDict) -> None: - v: dict[str, int] = x # OK - v.clear() # OK - - not_required_num_dict: IntDictWithNum = {"num": 1, "bar": 2} - regular_dict: dict[str, int] = not_required_num_dict # OK - f(not_required_num_dict) # OK - -In this case, methods that are previously unavailable on a TypedDict are allowed, -with signatures matching ``dict[str, VT]`` -(e.g.: ``__setitem__(self, key: str, value: VT) -> None``):: - - not_required_num_dict.clear() # OK - - reveal_type(not_required_num_dict.popitem()) # OK. Revealed type is 'tuple[str, int]' - - def f(not_required_num_dict: IntDictWithNum, key: str): - not_required_num_dict[key] = 42 # OK - del not_required_num_dict[key] # OK - -:ref:`Notes on indexed accesses ` from the previous section -still apply. - -``dict[str, VT]`` is not assignable to a TypedDict type, -because such dict can be a subtype of dict:: - - class CustomDict(dict[str, int]): - pass - - def f(might_not_be_a_builtin_dict: dict[str, int]): - int_dict: IntDict = might_not_be_a_builtin_dict # Not OK +Backwards Compatibility +----------------------- - not_a_builtin_dict = CustomDict({"num": 1}) - f(not_a_builtin_dict) +To retain backwards compatibility, type checkers should not infer a +TypedDict type unless it is sufficiently clear that this is desired by +the programmer. When unsure, an ordinary dictionary type should be +inferred. Otherwise existing code that type checks without errors may +start generating errors once TypedDict support is added to the type +checker, since TypedDict types are more restrictive than dictionary +types. In particular, they aren't subtypes of dictionary types. From eb914a40a7d8f24a2e67f4e5ea7ebb9e158eac88 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Sun, 17 Aug 2025 18:10:31 -0700 Subject: [PATCH 2/5] Apply suggestions from code review Co-authored-by: Joren Hammudoglu --- docs/spec/glossary.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/spec/glossary.rst b/docs/spec/glossary.rst index 36ef8da45..afbf9236b 100644 --- a/docs/spec/glossary.rst +++ b/docs/spec/glossary.rst @@ -62,7 +62,7 @@ This section defines a few terms that may be used elsewhere in the specification extra items A :ref:`TypedDict ` type with extra items may contain arbitrary additional :term:`items ` beyond those specified in the TypedDict definition, but those - items must be of the type specified by the TypedDict definition. + items must be of the type specified by that definition. A TypedDict with extra items can be created using the ``extra_items=`` argument to :py:func:`typing.TypedDict`. Extra items may or may not be :term:`read-only`. Compare :term:`closed` and :term:`open`. @@ -156,7 +156,7 @@ This section defines a few terms that may be used elsewhere in the specification read-only A read-only :term:`item` in a :ref:`TypedDict ` may not be modified. - Attempts to assign to or delete that item + Attempts to delete or assign to that item should be reported as type errors by a type checker. Read-only items are created using the :py:data:`typing.ReadOnly` qualifier. From 0f18c4b277211bc87726788fe49548a6cb293b8c Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Sun, 17 Aug 2025 18:19:17 -0700 Subject: [PATCH 3/5] feedback --- docs/spec/glossary.rst | 9 ++++----- docs/spec/typeddict.rst | 36 +++++++++++++++++++++--------------- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/docs/spec/glossary.rst b/docs/spec/glossary.rst index afbf9236b..1ab2e5f44 100644 --- a/docs/spec/glossary.rst +++ b/docs/spec/glossary.rst @@ -31,7 +31,7 @@ This section defines a few terms that may be used elsewhere in the specification A :ref:`TypedDict ` type is closed if it may not contain any additional :term:`items ` beyond those specified in the TypedDict definition. A closed TypedDict can be created using the ``closed=True`` argument to - :py:func:`typing.TypedDict`. + :py:func:`typing.TypedDict`, or equivalently by setting ``extra_items=Never``. Compare :term:`extra items` and :term:`open`. consistent @@ -100,10 +100,9 @@ This section defines a few terms that may be used elsewhere in the specification :pep:`3107` syntax (the filename ends in ``.py``). item - In the context of a :ref:`TypedDict `, an item is a key/value - pair defined in the TypedDict definition. Each item has a name (the key) - and a type (the value). Items may be :term:`required` or - :term:`non-required`, and may be :term:`read-only` or writable. + In the context of a :ref:`TypedDict `, an item consists of a name + (the dictionary key) and a type (representing the type that values corresponding to the key must have). + Items may be :term:`required` or :term:`non-required`, and may be :term:`read-only` or writable. materialize A :term:`gradual type` can be materialized to a more static type diff --git a/docs/spec/typeddict.rst b/docs/spec/typeddict.rst index bb2747457..2732f233e 100644 --- a/docs/spec/typeddict.rst +++ b/docs/spec/typeddict.rst @@ -17,7 +17,7 @@ TypedDict types can define any number of :term:`items `, which are string keys associated with values of a specified type. For example, a TypedDict may contain the item ``a: str``, indicating that the key ``a`` must map to a value of type ``str``. Items may be either :term:`required`, -meaning they must be present in any instance of the TypedDict type, or +meaning they must be present in every instance of the TypedDict type, or :term:`non-required`, meaning they may be omitted, but if they are present, they must be of the type specified in the TypedDict definition. By default, all items in a TypedDict are mutable, but items @@ -395,14 +395,19 @@ is inherited from its superclass by default:: pass However, subclasses may also explicitly use the ``closed`` and ``extra_items`` arguments -to change the openness of the TypedDict, but in some cases this yields a type checker error. -If the base class is open, all possible states are allowed in the subclass: it may remain open, -it may be closed (with ``closed=True``), or it may have extra items (with ``extra_items=...``). -If the base class is closed, any child classes must also be closed. -If the base class has extra items, but they are not read-only, the child class must also allow -the same extra items. If the base class has read-only extra items, the child class may be closed, -or it may redeclare its extra items with a type that is :term:`assignable` to the base class type. -Child classes may also have mutable extra items if the base class has read-only extra items. +to change the openness of the TypedDict, but in some cases this yields a type checker error: + +- If the base class is open, all possible states are allowed in the subclass: it may remain open, + it may be closed (with ``closed=True``), or it may have extra items (with ``extra_items=...``). + +- If the base class is closed, any child classes must also be closed. + +- If the base class has extra items, but they are not read-only, the child class must also allow + the same extra items. + +- If the base class has read-only extra items, the child class may be closed, + or it may redeclare its extra items with a type that is :term:`assignable` to the base class type. + Child classes may also have mutable extra items if the base class has read-only extra items. For example:: @@ -607,6 +612,7 @@ A TypedDict type is a subtype of ``dict[str, VT]`` if the following conditions a - The TypedDict type has mutable :term:`extra items` of a type that is :term:`equivalent` to ``VT``. - All items on the TypedDict satisfy the following conditions: + - The value type of the item is :term:`equivalent` to ``VT``. - The item is not read-only. - The item is not required. @@ -627,7 +633,7 @@ For example:: regular_dict: dict[str, int] = not_required_num_dict # OK f(not_required_num_dict) # OK -In this case, methods that are previously unavailable on a TypedDict are allowed, +In this case, some methods that are otherwise unavailable on a TypedDict are allowed, with signatures matching ``dict[str, VT]`` (e.g.: ``__setitem__(self, key: str, value: VT) -> None``):: @@ -742,7 +748,7 @@ This section discusses some specific operations in more detail. if the string value of ``e`` cannot be determined statically. (This simplifies to ``object`` if ``d`` is :term:`open`.) -* ``clear()`` is not safe on :term:`open` TypedDicts since it could remove required keys, some of which +* ``clear()`` is not safe on :term:`open` TypedDicts since it could remove required items, some of which may not be directly visible because of :term:`structural` :term:`assignability `. However, this method is safe on :term:`closed` TypedDicts and TypedDicts with :term:`extra items` if @@ -750,7 +756,7 @@ This section discusses some specific operations in more detail. or read-only items. * ``popitem()`` is similarly unsafe on many TypedDicts, even - if all known keys are not required (``total=False``). + if all known items are :term:`non-required`. * ``del obj['key']`` should be rejected unless ``'key'`` is a non-required, mutable key. @@ -785,14 +791,14 @@ This section discusses some specific operations in more detail. * The ``update()`` method should not allow mutating a read-only item. Therefore, type checkers should error if a TypedDict with a read-only item is updated with another TypedDict that declares - that key:: + that item:: class A(TypedDict): x: ReadOnly[int] y: int - a1: A = { "x": 1, "y": 2 } - a2: A = { "x": 3, "y": 4 } + a1: A = {"x": 1, "y": 2} + a2: A = {"x": 3, "y": 4} a1.update(a2) # Type check error: "x" is read-only in A Unless the declared value is of bottom type (:data:`~typing.Never`):: From 77ae2d19d9acd9d2351b52358e5d13dfda1ddffa Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Tue, 19 Aug 2025 14:53:05 -0700 Subject: [PATCH 4/5] Carl feedback --- docs/spec/concepts.rst | 8 ++++---- docs/spec/glossary.rst | 9 +++++++-- docs/spec/typeddict.rst | 4 +++- 3 files changed, 14 insertions(+), 7 deletions(-) diff --git a/docs/spec/concepts.rst b/docs/spec/concepts.rst index ee230f231..81fe481fe 100644 --- a/docs/spec/concepts.rst +++ b/docs/spec/concepts.rst @@ -69,7 +69,7 @@ attributes and/or methods. If an object ``v`` is a member of the set of objects denoted by a fully static type ``T``, we can say that ``v`` is a "member of" the type ``T``, or ``v`` -"inhabits" ``T``. +":term:`inhabits `" ``T``. Gradual types ~~~~~~~~~~~~~ @@ -298,9 +298,9 @@ visualize this analogy in the following table: * - ``B`` is :term:`equivalent` to ``A`` - ``B`` is :term:`consistent` with ``A`` -We can also define an **equivalence** relation on gradual types: the gradual -types ``A`` and ``B`` are equivalent (that is, the same gradual type, not -merely consistent with one another) if and only if all materializations of +We can also define an **equivalence** relation on gradual types: the gradual +types ``A`` and ``B`` are equivalent (that is, the same gradual type, not +merely consistent with one another) if and only if all materializations of ``A`` are also materializations of ``B``, and all materializations of ``B`` are also materializations of ``A``. diff --git a/docs/spec/glossary.rst b/docs/spec/glossary.rst index 1ab2e5f44..340dcbd4a 100644 --- a/docs/spec/glossary.rst +++ b/docs/spec/glossary.rst @@ -61,8 +61,8 @@ This section defines a few terms that may be used elsewhere in the specification extra items A :ref:`TypedDict ` type with extra items may contain arbitrary - additional :term:`items ` beyond those specified in the TypedDict definition, but those - items must be of the type specified by that definition. + additional key-value pairs beyond those specified in the TypedDict definition, but those + values must be of the type specified by the ``extra_items=`` argument to the definition. A TypedDict with extra items can be created using the ``extra_items=`` argument to :py:func:`typing.TypedDict`. Extra items may or may not be :term:`read-only`. Compare :term:`closed` and :term:`open`. @@ -94,6 +94,11 @@ This section defines a few terms that may be used elsewhere in the specification They can be :term:`materialized ` to a more static, or fully static, type. See :ref:`type-system-concepts`. + inhabit + A value is said to inhabit a type if it is a member of the set of values + represented by that type. For example, the value ``42`` inhabits the type + ``int``, and the value ``"hello"`` inhabits the type ``str``. + inline Inline type annotations are annotations that are included in the runtime code using :pep:`526` and diff --git a/docs/spec/typeddict.rst b/docs/spec/typeddict.rst index 2732f233e..c612a3f08 100644 --- a/docs/spec/typeddict.rst +++ b/docs/spec/typeddict.rst @@ -10,7 +10,7 @@ and ``NotRequired`` in :pep:`655`, use with ``Unpack`` in :pep:`692`, A TypedDict type represents ``dict`` objects that contain only keys of type ``str``. There are restrictions on which string keys are valid, and -which values can be associated with each key. Values that are members of a +which values can be associated with each key. Values that :term:`inhabit` a TypedDict type must be instances of ``dict`` itself, not a subclass. TypedDict types can define any number of :term:`items `, which are string @@ -371,10 +371,12 @@ Example:: class X(TypedDict): x: str y: ReadOnly[int] + z: int class Y(X): x: int # Type check error: cannot overwrite TypedDict field "x" y: bool # OK: bool is assignable to int, and a mutable item can override a read-only one + z: bool # Type check error: key is mutable, so subclass type must be consistent with superclass Openness ^^^^^^^^ From 9264235b225a290a20565d8250aa4d05a49a591a Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 25 Aug 2025 19:29:17 -0700 Subject: [PATCH 5/5] more clarity --- docs/spec/typeddict.rst | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/spec/typeddict.rst b/docs/spec/typeddict.rst index c612a3f08..1e0808f64 100644 --- a/docs/spec/typeddict.rst +++ b/docs/spec/typeddict.rst @@ -112,7 +112,8 @@ in the class definition: * ``closed``: a boolean literal (``True`` or ``False``) indicating whether the TypedDict is :term:`closed` (``True``) or :term:`open` (``False``). The latter is the default, except when inheriting from another TypedDict that - is not open (see :ref:`typeddict-inheritance`). + is not open (see :ref:`typeddict-inheritance`), or when the ``extra_items`` + argument is also used. As with ``total``, the value must be exactly ``True`` or ``False``. It is an error to use this argument together with ``extra_items=``. * ``extra_items``: indicates that the TypedDict has :term:`extra items`. The argument @@ -492,7 +493,7 @@ Multiple inheritance does not allow conflicting types for the same item:: Subtyping and assignability --------------------------- -Because TypedDict types are :term:`structural` types, a TypedDict ``T1`` is assignable to another +Because TypedDict types are :term:`structural` types, a TypedDict ``T1`` is :term:`assignable` to another TypedDict type ``T2`` if the two are structurally compatible, meaning that all operations that are allowed on ``T2`` are also allowed on ``T1``. For similar reasons, TypedDict types are generally not assignable to any specialization of ``dict`` or ``Mapping``, other than ``Mapping[str, object]``, @@ -501,7 +502,10 @@ to these types. The rest of this section discusses the :term:`subtyping ` rules for TypedDict in more detail. As with any type, the rules for :term:`assignability ` can be derived from the subtyping -rules using the :term:`materialization ` procedure. +rules using the :term:`materialization ` procedure. Generally, this means that where +":term:`equivalent`" is mentioned below, the :term:`consistency ` relation can be used instead +when implementing assignability, and where ":term:`subtyping `" between elements of a +TypedDict is mentioned, assignability can be used instead when implementing assignability between TypedDicts. Subtyping between TypedDict types ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -518,6 +522,8 @@ The conditions are as follows: - It must also be required in ``B``. - If it is read-only in ``A``, the item type in ``B`` must be a subtype of the item type in ``A``. + (For :term:`assignability ` between two TypedDicts, the first item must instead + be assignable to the second.) - If it is mutable in ``A``, it must also be mutable in ``B``, and the item type in ``B`` must be :term:`equivalent` to the item type in ``A``. (It follows that for assignability, the two item types @@ -536,14 +542,14 @@ The conditions are as follows: - If it is mutable in ``A``: - If ``B`` has an item with the same key, it must also be mutable, and its item type must be - :term:`equivalent` to the item type in ``A``. (As before, it follows that for assignability, the two item types - must be :term:`consistent`.) + :term:`equivalent` to the item type in ``A``. - Else: - If ``B`` is closed, the check fails. - If ``B`` has extra items, the extra items type must not be read-only and must be :term:`equivalent` to the item type in ``A``. + - If ``A`` is closed, ``B`` must also be closed, and it must not contain any items that are not present in ``A``. - If ``A`` has read-only extra items, ``B`` must either be closed or also have extra items, and the extra items type in ``B`` must be a subtype of the extra items type in ``A``. Additionally, for any items in ``B`` that are not present in ``A``,