Skip to content

gh-116127: PEP-705: Add ReadOnly support for TypedDict #116350

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions Doc/library/typing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1275,6 +1275,26 @@ These can be used as types in annotations. They all support subscription using

.. versionadded:: 3.11

.. data:: ReadOnly

A special typing construct to mark an item of a :class:`TypedDict` as read-only.

For example::

class Movie(TypedDict):
title: ReadOnly[str]
year: int

def mutate_movie(m: Movie) -> None:
m["year"] = 1992 # allowed
m["title"] = "The Matrix" # typechecker error

There is no runtime checking for this property.

See :class:`TypedDict` and :pep:`705` for more details.

.. versionadded:: 3.13

.. data:: Annotated

Special typing form to add context-specific metadata to an annotation.
Expand Down Expand Up @@ -2455,6 +2475,22 @@ types.
``__required_keys__`` and ``__optional_keys__`` rely on may not work
properly, and the values of the attributes may be incorrect.

Support for :data:`ReadOnly` is reflected in the following attributes::

.. attribute:: __readonly_keys__

A :class:`frozenset` containing the names of all read-only keys. Keys
are read-only if they carry the :data:`ReadOnly` qualifier.

.. versionadded:: 3.13

.. attribute:: __mutable_keys__

A :class:`frozenset` containing the names of all mutable keys. Keys
are mutable if they do not carry the :data:`ReadOnly` qualifier.

.. versionadded:: 3.13

See :pep:`589` for more examples and detailed rules of using ``TypedDict``.

.. versionadded:: 3.8
Expand All @@ -2469,6 +2505,9 @@ types.
.. versionchanged:: 3.13
Removed support for the keyword-argument method of creating ``TypedDict``\ s.

.. versionchanged:: 3.13
Support for the :data:`ReadOnly` qualifier was added.

.. deprecated-removed:: 3.13 3.15
When using the functional syntax to create a TypedDict class, failing to
pass a value to the 'fields' parameter (``TD = TypedDict("TD")``) is
Expand Down
4 changes: 4 additions & 0 deletions Doc/whatsnew/3.13.rst
Original file line number Diff line number Diff line change
Expand Up @@ -582,6 +582,10 @@ typing
check whether a class is a :class:`typing.Protocol`. (Contributed by Jelle Zijlstra in
:gh:`104873`.)

* Add :data:`typing.ReadOnly`, a special typing construct to mark
an item of a :class:`typing.TypedDict` as read-only for type checkers.
See :pep:`705` for more details.

unicodedata
-----------

Expand Down
65 changes: 64 additions & 1 deletion Lib/test/test_typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
from typing import dataclass_transform
from typing import no_type_check, no_type_check_decorator
from typing import Type
from typing import NamedTuple, NotRequired, Required, TypedDict
from typing import NamedTuple, NotRequired, Required, ReadOnly, TypedDict
from typing import IO, TextIO, BinaryIO
from typing import Pattern, Match
from typing import Annotated, ForwardRef
Expand Down Expand Up @@ -8334,6 +8334,69 @@ class T4(TypedDict, Generic[S]): pass
self.assertEqual(klass.__optional_keys__, set())
self.assertIsInstance(klass(), dict)

def test_readonly_inheritance(self):
class Base1(TypedDict):
a: ReadOnly[int]

class Child1(Base1):
b: str

self.assertEqual(Child1.__readonly_keys__, frozenset({'a'}))
self.assertEqual(Child1.__mutable_keys__, frozenset({'b'}))

class Base2(TypedDict):
a: ReadOnly[int]

class Child2(Base2):
b: str

self.assertEqual(Child1.__readonly_keys__, frozenset({'a'}))
self.assertEqual(Child1.__mutable_keys__, frozenset({'b'}))

def test_cannot_make_mutable_key_readonly(self):
class Base(TypedDict):
a: int

with self.assertRaises(TypeError):
class Child(Base):
a: ReadOnly[int]

def test_can_make_readonly_key_mutable(self):
class Base(TypedDict):
a: ReadOnly[int]

class Child(Base):
a: int

self.assertEqual(Child.__readonly_keys__, frozenset())
self.assertEqual(Child.__mutable_keys__, frozenset({'a'}))

def test_combine_qualifiers(self):
class AllTheThings(TypedDict):
a: Annotated[Required[ReadOnly[int]], "why not"]
b: Required[Annotated[ReadOnly[int], "why not"]]
c: ReadOnly[NotRequired[Annotated[int, "why not"]]]
d: NotRequired[Annotated[int, "why not"]]

self.assertEqual(AllTheThings.__required_keys__, frozenset({'a', 'b'}))
self.assertEqual(AllTheThings.__optional_keys__, frozenset({'c', 'd'}))
self.assertEqual(AllTheThings.__readonly_keys__, frozenset({'a', 'b', 'c'}))
self.assertEqual(AllTheThings.__mutable_keys__, frozenset({'d'}))

self.assertEqual(
get_type_hints(AllTheThings, include_extras=False),
{'a': int, 'b': int, 'c': int, 'd': int},
)
self.assertEqual(
get_type_hints(AllTheThings, include_extras=True),
{
'a': Annotated[Required[ReadOnly[int]], 'why not'],
'b': Required[Annotated[ReadOnly[int], 'why not']],
'c': ReadOnly[NotRequired[Annotated[int, 'why not']]],
'd': NotRequired[Annotated[int, 'why not']],
},
)


class RequiredTests(BaseTestCase):

Expand Down
83 changes: 73 additions & 10 deletions Lib/typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,7 @@
'override',
'ParamSpecArgs',
'ParamSpecKwargs',
'ReadOnly',
'Required',
'reveal_type',
'runtime_checkable',
Expand Down Expand Up @@ -2301,7 +2302,7 @@ def _strip_annotations(t):
"""Strip the annotations from a given type."""
if isinstance(t, _AnnotatedAlias):
return _strip_annotations(t.__origin__)
if hasattr(t, "__origin__") and t.__origin__ in (Required, NotRequired):
if hasattr(t, "__origin__") and t.__origin__ in (Required, NotRequired, ReadOnly):
return _strip_annotations(t.__args__[0])
if isinstance(t, _GenericAlias):
stripped_args = tuple(_strip_annotations(a) for a in t.__args__)
Expand Down Expand Up @@ -2922,6 +2923,28 @@ def _namedtuple_mro_entries(bases):
NamedTuple.__mro_entries__ = _namedtuple_mro_entries


def _get_typeddict_qualifiers(annotation_type):
while True:
annotation_origin = get_origin(annotation_type)
if annotation_origin is Annotated:
annotation_args = get_args(annotation_type)
if annotation_args:
annotation_type = annotation_args[0]
else:
break
elif annotation_origin is Required:
yield Required
(annotation_type,) = get_args(annotation_type)
elif annotation_origin is NotRequired:
yield NotRequired
(annotation_type,) = get_args(annotation_type)
elif annotation_origin is ReadOnly:
yield ReadOnly
(annotation_type,) = get_args(annotation_type)
else:
break


class _TypedDictMeta(type):
def __new__(cls, name, bases, ns, total=True):
"""Create a new typed dict class object.
Expand Down Expand Up @@ -2955,6 +2978,8 @@ def __new__(cls, name, bases, ns, total=True):
}
required_keys = set()
optional_keys = set()
readonly_keys = set()
mutable_keys = set()

for base in bases:
annotations.update(base.__dict__.get('__annotations__', {}))
Expand All @@ -2967,18 +2992,15 @@ def __new__(cls, name, bases, ns, total=True):
required_keys -= base_optional
optional_keys |= base_optional

readonly_keys.update(base.__dict__.get('__readonly_keys__', ()))
mutable_keys.update(base.__dict__.get('__mutable_keys__', ()))

annotations.update(own_annotations)
for annotation_key, annotation_type in own_annotations.items():
annotation_origin = get_origin(annotation_type)
if annotation_origin is Annotated:
annotation_args = get_args(annotation_type)
if annotation_args:
annotation_type = annotation_args[0]
annotation_origin = get_origin(annotation_type)

if annotation_origin is Required:
qualifiers = set(_get_typeddict_qualifiers(annotation_type))
if Required in qualifiers:
is_required = True
elif annotation_origin is NotRequired:
elif NotRequired in qualifiers:
is_required = False
else:
is_required = total
Expand All @@ -2990,13 +3012,26 @@ def __new__(cls, name, bases, ns, total=True):
optional_keys.add(annotation_key)
required_keys.discard(annotation_key)

if ReadOnly in qualifiers:
if annotation_key in mutable_keys:
raise TypeError(
f"Cannot override mutable key {annotation_key!r}"
" with read-only key"
)
readonly_keys.add(annotation_key)
else:
mutable_keys.add(annotation_key)
readonly_keys.discard(annotation_key)

assert required_keys.isdisjoint(optional_keys), (
f"Required keys overlap with optional keys in {name}:"
f" {required_keys=}, {optional_keys=}"
)
tp_dict.__annotations__ = annotations
tp_dict.__required_keys__ = frozenset(required_keys)
tp_dict.__optional_keys__ = frozenset(optional_keys)
tp_dict.__readonly_keys__ = frozenset(readonly_keys)
tp_dict.__mutable_keys__ = frozenset(mutable_keys)
tp_dict.__total__ = total
return tp_dict

Expand Down Expand Up @@ -3055,6 +3090,14 @@ class Point2D(TypedDict):
y: NotRequired[int] # the "y" key can be omitted

See PEP 655 for more details on Required and NotRequired.

The ReadOnly special form can be used
to mark individual keys as immutable for type checkers::

class DatabaseUser(TypedDict):
id: ReadOnly[int] # the "id" key must not be modified
username: str # the "username" key can be changed

"""
if fields is _sentinel or fields is None:
import warnings
Expand Down Expand Up @@ -3131,6 +3174,26 @@ class Movie(TypedDict):
return _GenericAlias(self, (item,))


@_SpecialForm
def ReadOnly(self, parameters):
"""A special typing construct to mark an item of a TypedDict as read-only.

For example::

class Movie(TypedDict):
title: ReadOnly[str]
year: int

def mutate_movie(m: Movie) -> None:
m["year"] = 1992 # allowed
m["title"] = "The Matrix" # typechecker error

There is no runtime checking for this property.
"""
item = _type_check(parameters, f'{self._name} accepts only a single type.')
return _GenericAlias(self, (item,))


class NewType:
"""NewType creates simple unique types with almost zero runtime overhead.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
:mod:`typing`: implement :pep:`705` which adds :data:`typing.ReadOnly`
support to :class:`typing.TypedDict`.