Skip to content

gh-116126: Implement PEP 696 #116129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 56 commits into from
May 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
e7ca092
gh-116126: Grammar changes for PEP 696
JelleZijlstra Feb 29, 2024
828679e
Fix test_ast
JelleZijlstra Feb 29, 2024
d3b472b
new tests
JelleZijlstra Feb 29, 2024
6e467ee
fix test_unparse
JelleZijlstra Feb 29, 2024
fa48580
possible solution
JelleZijlstra Mar 1, 2024
49b077b
Can't use the name either
JelleZijlstra Mar 1, 2024
5cfd5e2
blurb
JelleZijlstra Mar 1, 2024
01b3270
should not have pushed that
JelleZijlstra Mar 1, 2024
7a69204
maybe this will compile
JelleZijlstra Mar 1, 2024
a265008
Feature version support
JelleZijlstra Mar 1, 2024
0c28297
Merge remote-tracking branch 'upstream/main' into pep696v2
JelleZijlstra Apr 10, 2024
8aa5b27
default_ -> default_value
JelleZijlstra Apr 10, 2024
bdd01bb
ast docs
JelleZijlstra Apr 10, 2024
2c28b9a
more docs
JelleZijlstra Apr 10, 2024
b0d467f
Fix typevarobject.c
JelleZijlstra Apr 10, 2024
d7e91e2
default=<unrepresentable>
JelleZijlstra Apr 10, 2024
35226b4
NoneType is immortal
JelleZijlstra Apr 10, 2024
2e079ee
Runtime implementation
JelleZijlstra Apr 10, 2024
e037052
What's New
JelleZijlstra Apr 10, 2024
7b15f11
Document None/NoneType weirdness
JelleZijlstra Apr 10, 2024
11bd102
fix doctest
JelleZijlstra Apr 11, 2024
c54aeed
Merge remote-tracking branch 'upstream/main' into pep696v2
JelleZijlstra Apr 23, 2024
71a348b
Feedback from Alex
JelleZijlstra Apr 23, 2024
900cd40
Partial work
JelleZijlstra Apr 23, 2024
c13420b
Allow * in TypeVarTuple default
JelleZijlstra Apr 23, 2024
8a08c62
Test and fix grammar
JelleZijlstra Apr 23, 2024
613159f
Fix TypeVarTuple substitution
JelleZijlstra Apr 23, 2024
4e0435e
Two more cases
JelleZijlstra Apr 23, 2024
7d54ace
Merge remote-tracking branch 'upstream/main' into pep696v2
JelleZijlstra Apr 23, 2024
12581d8
Fix TypeVar substitution
JelleZijlstra Apr 23, 2024
2b5a102
Fix ParamSpec
JelleZijlstra Apr 23, 2024
a2c48c9
Prohibit default after TypeVarTuple
JelleZijlstra Apr 23, 2024
29b9435
Wrap new syntax in run_code()
JelleZijlstra Apr 23, 2024
326de17
Silence Ruff
JelleZijlstra Apr 23, 2024
6f66775
Fix parameter markup
JelleZijlstra Apr 23, 2024
7efa4ce
default_ -> default_value
JelleZijlstra Apr 23, 2024
29ad843
Fix errors
JelleZijlstra Apr 23, 2024
2c04c14
arguments, not parameters
JelleZijlstra Apr 23, 2024
7d4f3fd
Merge remote-tracking branch 'upstream/main' into pep696v2
JelleZijlstra Apr 23, 2024
1a14367
Apply suggestions from code review
JelleZijlstra Apr 23, 2024
5890d79
regen clinic
JelleZijlstra Apr 23, 2024
19a7b48
variadic args for _unpack_args
JelleZijlstra Apr 23, 2024
ef0616b
Update Lib/typing.py
JelleZijlstra Apr 23, 2024
9d4e842
Update Doc/reference/executionmodel.rst
JelleZijlstra Apr 23, 2024
2686b97
Roundtrip tests
JelleZijlstra Apr 24, 2024
98d1dec
Merge remote-tracking branch 'upstream/main' into pep696v2
JelleZijlstra Apr 28, 2024
fbb6405
Fix doctest
JelleZijlstra Apr 28, 2024
e0ccfeb
Add typing.NoDefault as the default for TypeVar's default
JelleZijlstra Apr 28, 2024
15aaff9
Apply suggestions from code review
JelleZijlstra Apr 28, 2024
86f5aed
Add .has_default() and update docs
JelleZijlstra Apr 28, 2024
3e0c0fc
Apply suggestions from code review
JelleZijlstra Apr 28, 2024
f5a3d4b
remove trailing space
JelleZijlstra Apr 28, 2024
92ea108
Merge remote-tracking branch 'upstream/main' into pep696v2
JelleZijlstra Apr 30, 2024
5f6fdfd
Ignore new C globals
JelleZijlstra Apr 30, 2024
b3f053c
Merge remote-tracking branch 'upstream/main' into pep696v2
JelleZijlstra May 3, 2024
8c3e0b4
Fix scoping key
JelleZijlstra May 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 38 additions & 14 deletions Doc/library/ast.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1748,43 +1748,57 @@ Type parameters
:ref:`Type parameters <type-params>` can exist on classes, functions, and type
aliases.

.. class:: TypeVar(name, bound)
.. class:: TypeVar(name, bound, default_value)

A :class:`typing.TypeVar`. ``name`` is the name of the type variable.
``bound`` is the bound or constraints, if any. If ``bound`` is a :class:`Tuple`,
it represents constraints; otherwise it represents the bound.
A :class:`typing.TypeVar`. *name* is the name of the type variable.
*bound* is the bound or constraints, if any. If *bound* is a :class:`Tuple`,
it represents constraints; otherwise it represents the bound. *default_value*
is the default value; if the :class:`!TypeVar` has no default, this
attribute will be set to ``None``.

.. doctest::

>>> print(ast.dump(ast.parse("type Alias[T: int] = list[T]"), indent=4))
>>> print(ast.dump(ast.parse("type Alias[T: int = bool] = list[T]"), indent=4))
Module(
body=[
TypeAlias(
name=Name(id='Alias', ctx=Store()),
type_params=[
TypeVar(
name='T',
bound=Name(id='int', ctx=Load()))],
bound=Name(id='int', ctx=Load()),
default_value=Name(id='bool', ctx=Load()))],
value=Subscript(
value=Name(id='list', ctx=Load()),
slice=Name(id='T', ctx=Load()),
ctx=Load()))])

.. versionadded:: 3.12

.. class:: ParamSpec(name)
.. versionchanged:: 3.13
Added the *default_value* parameter.

.. class:: ParamSpec(name, default_value)

A :class:`typing.ParamSpec`. ``name`` is the name of the parameter specification.
A :class:`typing.ParamSpec`. *name* is the name of the parameter specification.
*default_value* is the default value; if the :class:`!ParamSpec` has no default,
this attribute will be set to ``None``.

.. doctest::

>>> print(ast.dump(ast.parse("type Alias[**P] = Callable[P, int]"), indent=4))
>>> print(ast.dump(ast.parse("type Alias[**P = (int, str)] = Callable[P, int]"), indent=4))
Module(
body=[
TypeAlias(
name=Name(id='Alias', ctx=Store()),
type_params=[
ParamSpec(name='P')],
ParamSpec(
name='P',
default_value=Tuple(
elts=[
Name(id='int', ctx=Load()),
Name(id='str', ctx=Load())],
ctx=Load()))],
value=Subscript(
value=Name(id='Callable', ctx=Load()),
slice=Tuple(
Expand All @@ -1796,19 +1810,26 @@ aliases.

.. versionadded:: 3.12

.. class:: TypeVarTuple(name)
.. versionchanged:: 3.13
Added the *default_value* parameter.

.. class:: TypeVarTuple(name, default_value)

A :class:`typing.TypeVarTuple`. ``name`` is the name of the type variable tuple.
A :class:`typing.TypeVarTuple`. *name* is the name of the type variable tuple.
*default_value* is the default value; if the :class:`!TypeVarTuple` has no
default, this attribute will be set to ``None``.

.. doctest::

>>> print(ast.dump(ast.parse("type Alias[*Ts] = tuple[*Ts]"), indent=4))
>>> print(ast.dump(ast.parse("type Alias[*Ts = ()] = tuple[*Ts]"), indent=4))
Module(
body=[
TypeAlias(
name=Name(id='Alias', ctx=Store()),
type_params=[
TypeVarTuple(name='Ts')],
TypeVarTuple(
name='Ts',
default_value=Tuple(ctx=Load()))],
value=Subscript(
value=Name(id='tuple', ctx=Load()),
slice=Tuple(
Expand All @@ -1821,6 +1842,9 @@ aliases.

.. versionadded:: 3.12

.. versionchanged:: 3.13
Added the *default_value* parameter.

Function and class definitions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down
82 changes: 79 additions & 3 deletions Doc/library/typing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1616,7 +1616,7 @@ without the dedicated syntax, as documented below.

.. _typevar:

.. class:: TypeVar(name, *constraints, bound=None, covariant=False, contravariant=False, infer_variance=False)
.. class:: TypeVar(name, *constraints, bound=None, covariant=False, contravariant=False, infer_variance=False, default=typing.NoDefault)

Type variable.

Expand Down Expand Up @@ -1754,15 +1754,35 @@ without the dedicated syntax, as documented below.
the constraints are evaluated only when the attribute is accessed, not when
the type variable is created (see :ref:`lazy-evaluation`).

.. attribute:: __default__

The default value of the type variable, or :data:`typing.NoDefault` if it
has no default.

.. versionadded:: 3.13

.. method:: has_default()

Return whether or not the type variable has a default value. This is equivalent
to checking whether :attr:`__default__` is not the :data:`typing.NoDefault`
singleton, except that it does not force evaluation of the
:ref:`lazily evaluated <lazy-evaluation>` default value.

.. versionadded:: 3.13

.. versionchanged:: 3.12

Type variables can now be declared using the
:ref:`type parameter <type-params>` syntax introduced by :pep:`695`.
The ``infer_variance`` parameter was added.

.. versionchanged:: 3.13

Support for default values was added.

.. _typevartuple:

.. class:: TypeVarTuple(name)
.. class:: TypeVarTuple(name, default=typing.NoDefault)

Type variable tuple. A specialized form of :ref:`type variable <typevar>`
that enables *variadic* generics.
Expand Down Expand Up @@ -1872,14 +1892,34 @@ without the dedicated syntax, as documented below.

The name of the type variable tuple.

.. attribute:: __default__

The default value of the type variable tuple, or :data:`typing.NoDefault` if it
has no default.

.. versionadded:: 3.13

.. method:: has_default()

Return whether or not the type variable tuple has a default value. This is equivalent
to checking whether :attr:`__default__` is not the :data:`typing.NoDefault`
singleton, except that it does not force evaluation of the
:ref:`lazily evaluated <lazy-evaluation>` default value.

.. versionadded:: 3.13

.. versionadded:: 3.11

.. versionchanged:: 3.12

Type variable tuples can now be declared using the
:ref:`type parameter <type-params>` syntax introduced by :pep:`695`.

.. class:: ParamSpec(name, *, bound=None, covariant=False, contravariant=False)
.. versionchanged:: 3.13

Support for default values was added.

.. class:: ParamSpec(name, *, bound=None, covariant=False, contravariant=False, default=typing.NoDefault)

Parameter specification variable. A specialized version of
:ref:`type variables <typevar>`.
Expand Down Expand Up @@ -1948,6 +1988,22 @@ without the dedicated syntax, as documented below.

The name of the parameter specification.

.. attribute:: __default__

The default value of the parameter specification, or :data:`typing.NoDefault` if it
has no default.

.. versionadded:: 3.13

.. method:: has_default()

Return whether or not the parameter specification has a default value. This is equivalent
to checking whether :attr:`__default__` is not the :data:`typing.NoDefault`
singleton, except that it does not force evaluation of the
:ref:`lazily evaluated <lazy-evaluation>` default value.

.. versionadded:: 3.13

Parameter specification variables created with ``covariant=True`` or
``contravariant=True`` can be used to declare covariant or contravariant
generic types. The ``bound`` argument is also accepted, similar to
Expand All @@ -1961,6 +2017,10 @@ without the dedicated syntax, as documented below.
Parameter specifications can now be declared using the
:ref:`type parameter <type-params>` syntax introduced by :pep:`695`.

.. versionchanged:: 3.13

Support for default values was added.

.. note::
Only parameter specification variables defined in global scope can
be pickled.
Expand Down Expand Up @@ -3173,6 +3233,22 @@ Introspection helpers

.. versionadded:: 3.7.4

.. data:: NoDefault

A sentinel object used to indicate that a type parameter has no default
value. For example:

.. doctest::

>>> T = TypeVar("T")
>>> T.__default__ is typing.NoDefault
True
>>> S = TypeVar("S", default=None)
>>> S.__default__ is None
True
Comment on lines +3244 to +3248
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>>> T.__default__ is typing.NoDefault
True
>>> S = TypeVar("S", default=None)
>>> S.__default__ is None
True
>>> T.has_default()
False
>>> T.__default__ is typing.NoDefault
True
>>> S = TypeVar("S", default=None)
>>> S.has_default()
True
>>> S.__default__ is None
True

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought of that but it felt out of place in the docs for NoDefault, since the cases you added don't use NoDefault at all.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough -- I thought it was quite nice to see together in one example how the two concepts interrelate, but I definitely don't feel strongly!


.. versionadded:: 3.13

Constant
--------

Expand Down
31 changes: 23 additions & 8 deletions Doc/reference/compound_stmts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1620,15 +1620,18 @@ Type parameter lists

.. versionadded:: 3.12

.. versionchanged:: 3.13
Support for default values was added (see :pep:`696`).

.. index::
single: type parameters

.. productionlist:: python-grammar
type_params: "[" `type_param` ("," `type_param`)* "]"
type_param: `typevar` | `typevartuple` | `paramspec`
typevar: `identifier` (":" `expression`)?
typevartuple: "*" `identifier`
paramspec: "**" `identifier`
typevar: `identifier` (":" `expression`)? ("=" `expression`)?
typevartuple: "*" `identifier` ("=" `expression`)?
paramspec: "**" `identifier` ("=" `expression`)?

:ref:`Functions <def>` (including :ref:`coroutines <async def>`),
:ref:`classes <class>` and :ref:`type aliases <type>` may
Expand Down Expand Up @@ -1694,19 +1697,31 @@ evaluated in a separate :ref:`annotation scope <annotation-scopes>`.
:data:`typing.TypeVarTuple`\ s and :data:`typing.ParamSpec`\ s cannot have bounds
or constraints.

All three flavors of type parameters can also have a *default value*, which is used
when the type parameter is not explicitly provided. This is added by appending
a single equals sign (``=``) followed by an expression. Like the bounds and
constraints of type variables, the default value is not evaluated when the
object is created, but only when the type parameter's ``__default__`` attribute
is accessed. To this end, the default value is evaluated in a separate
:ref:`annotation scope <annotation-scopes>`. If no default value is specified
for a type parameter, the ``__default__`` attribute is set to the special
sentinel object :data:`typing.NoDefault`.

The following example indicates the full set of allowed type parameter declarations::

def overly_generic[
SimpleTypeVar,
TypeVarWithDefault = int,
TypeVarWithBound: int,
TypeVarWithConstraints: (str, bytes),
*SimpleTypeVarTuple,
**SimpleParamSpec,
*SimpleTypeVarTuple = (int, float),
**SimpleParamSpec = (str, bytearray),
](
a: SimpleTypeVar,
b: TypeVarWithBound,
c: Callable[SimpleParamSpec, TypeVarWithConstraints],
*d: SimpleTypeVarTuple,
b: TypeVarWithDefault,
c: TypeVarWithBound,
d: Callable[SimpleParamSpec, TypeVarWithConstraints],
*e: SimpleTypeVarTuple,
): ...

.. _generic-functions:
Expand Down
8 changes: 6 additions & 2 deletions Doc/reference/executionmodel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ Annotation scopes are used in the following contexts:
* Type parameter lists for :ref:`generic classes <generic-classes>`.
A generic class's base classes and
keyword arguments are executed within the annotation scope, but its decorators are not.
* The bounds and constraints for type variables
* The bounds, constraints, and default values for type parameters
(:ref:`lazily evaluated <lazy-evaluation>`).
* The value of type aliases (:ref:`lazily evaluated <lazy-evaluation>`).

Expand All @@ -232,13 +232,17 @@ Annotation scopes differ from function scopes in the following ways:
.. versionadded:: 3.12
Annotation scopes were introduced in Python 3.12 as part of :pep:`695`.

.. versionchanged:: 3.13
Annotation scopes are also used for type parameter defaults, as
introduced by :pep:`696`.

.. _lazy-evaluation:

Lazy evaluation
---------------

The values of type aliases created through the :keyword:`type` statement are
*lazily evaluated*. The same applies to the bounds and constraints of type
*lazily evaluated*. The same applies to the bounds, constraints, and default values of type
variables created through the :ref:`type parameter syntax <type-params>`.
This means that they are not evaluated when the type alias or type variable is
created. Instead, they are only evaluated when doing so is necessary to resolve
Expand Down
6 changes: 6 additions & 0 deletions Doc/whatsnew/3.13.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,8 @@ Interpreter improvements:

New typing features:

* :pep:`696`: Type parameters (:data:`typing.TypeVar`, :data:`typing.ParamSpec`,
and :data:`typing.TypeVarTuple`) now support defaults.
Comment on lines +92 to +93
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a very short description of the changes, considering that it involves a change to CPython's grammar 😃 but we can add a more expansive description later; this is non-blocking

* :pep:`742`: :data:`typing.TypeIs` was added, providing more intuitive
type narrowing behavior.

Expand Down Expand Up @@ -850,6 +852,10 @@ typing
an item of a :class:`typing.TypedDict` as read-only for type checkers.
See :pep:`705` for more details.

* Add :data:`typing.NoDefault`, a sentinel object used to represent the defaults
of some parameters in the :mod:`typing` module. (Contributed by Jelle Zijlstra in
:gh:`116126`.)

unicodedata
-----------

Expand Down
10 changes: 7 additions & 3 deletions Grammar/python.gram
Original file line number Diff line number Diff line change
Expand Up @@ -647,21 +647,25 @@ type_params[asdl_type_param_seq*]: '[' t=type_param_seq ']' {
type_param_seq[asdl_type_param_seq*]: a[asdl_type_param_seq*]=','.type_param+ [','] { a }

type_param[type_param_ty] (memo):
| a=NAME b=[type_param_bound] { _PyAST_TypeVar(a->v.Name.id, b, EXTRA) }
| a=NAME b=[type_param_bound] c=[type_param_default] { _PyAST_TypeVar(a->v.Name.id, b, c, EXTRA) }
| '*' a=NAME colon=':' e=expression {
RAISE_SYNTAX_ERROR_STARTING_FROM(colon, e->kind == Tuple_kind
? "cannot use constraints with TypeVarTuple"
: "cannot use bound with TypeVarTuple")
}
| '*' a=NAME { _PyAST_TypeVarTuple(a->v.Name.id, EXTRA) }
| '*' a=NAME b=[type_param_starred_default] { _PyAST_TypeVarTuple(a->v.Name.id, b, EXTRA) }
| '**' a=NAME colon=':' e=expression {
RAISE_SYNTAX_ERROR_STARTING_FROM(colon, e->kind == Tuple_kind
? "cannot use constraints with ParamSpec"
: "cannot use bound with ParamSpec")
}
| '**' a=NAME { _PyAST_ParamSpec(a->v.Name.id, EXTRA) }
| '**' a=NAME b=[type_param_default] { _PyAST_ParamSpec(a->v.Name.id, b, EXTRA) }

type_param_bound[expr_ty]: ':' e=expression { e }
type_param_default[expr_ty]: '=' e=expression {
CHECK_VERSION(expr_ty, 13, "Type parameter defaults are", e) }
type_param_starred_default[expr_ty]: '=' e=star_expression {
CHECK_VERSION(expr_ty, 13, "Type parameter defaults are", e) }

# EXPRESSIONS
# -----------
Expand Down
Loading
Loading