gh-74929: PEP 667 general docs update

* expand on What's New entry for PEP 667 * review (and update) all mentions of "locals" in documentation C API documentation updates will be added in a separate PR
python · ncoghlan · May 21, 2024 · May 20, 2024 · May 20, 2024 · May 20, 2024
commit 6ecb4b33e2b533ee46d2485d3a3d2c73df1e1b5e
diff --git a/Doc/c-api/frame.rst b/Doc/c-api/frame.rst
@@ -123,8 +123,9 @@ See also :ref:`Reflection <reflection>`.
    Get the *frame*'s :attr:`~frame.f_locals` attribute.
    If the frame refers to a function or comprehension, this returns
    a write-through proxy object that allows modifying the locals.
-   In all other cases (classes, modules) it returns the :class:`dict`
-   representing the frame locals directly.
+   In all other cases (classes, modules, :func:`exec`, :func:`eval`) it returns
+   the mapping representing the frame locals directly (as described for
+   :func:`locals`).
 
    Return a :term:`strong reference`.
 

diff --git a/Doc/library/code.rst b/Doc/library/code.rst
@@ -18,9 +18,9 @@ build applications which provide an interactive interpreter prompt.
    This class deals with parsing and interpreter state (the user's namespace); it
    does not deal with input buffering or prompting or input file naming (the
    filename is always passed in explicitly). The optional *locals* argument
-   specifies the dictionary in which code will be executed; it defaults to a newly
-   created dictionary with key ``'__name__'`` set to ``'__console__'`` and key
-   ``'__doc__'`` set to ``None``.
+   specifies the mapping namespace in which code will be executed; it defaults
+   to a newly created dictionary with key ``'__name__'`` set to
+   ``'__console__'`` and key ``'__doc__'`` set to ``None``.
 
 
 .. class:: InteractiveConsole(locals=None, filename="<console>", local_exit=False)

diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
@@ -543,18 +543,19 @@ are always available.  They are listed here in alphabetical order.
 
    The *expression* argument is parsed and evaluated as a Python expression
    (technically speaking, a condition list) using the *globals* and *locals*
-   dictionaries as global and local namespace.  If the *globals* dictionary is
+   mappings as global and local namespace.  If the *globals* dictionary is
    present and does not contain a value for the key ``__builtins__``, a
    reference to the dictionary of the built-in module :mod:`builtins` is
    inserted under that key before *expression* is parsed.  That way you can
    control what builtins are available to the executed code by inserting your
    own ``__builtins__`` dictionary into *globals* before passing it to
-   :func:`eval`.  If the *locals* dictionary is omitted it defaults to the
-   *globals* dictionary.  If both dictionaries are omitted, the expression is
+   :func:`eval`.  If the *locals* mapping is omitted it defaults to the
+   *globals* dictionary.  If both mappings are omitted, the expression is
    executed with the *globals* and *locals* in the environment where
-   :func:`eval` is called.  Note, *eval()* does not have access to the
+   :func:`eval` is called.  Note, *eval()* will only have access to the
    :term:`nested scopes <nested scope>` (non-locals) in the enclosing
-   environment.
+   environment if they are already referenced in the scope that is calling
+   :func:`eval` (e.g. via a :keyword:`nonlocal` statement).
 
    Example:
 
@@ -587,6 +588,11 @@ are always available.  They are listed here in alphabetical order.
 
       The *globals* and *locals* arguments can now be passed as keywords.
 
+   .. versionchanged:: 3.13
+
+      The semantics of the default *locals* namespace have been adjusted as
+      described for the :func:`locals` builtin.
+
 .. index:: pair: built-in function; exec
 
 .. function:: exec(source, /, globals=None, locals=None, *, closure=None)
@@ -608,9 +614,9 @@ are always available.  They are listed here in alphabetical order.
    will be used for both the global and the local variables.  If *globals* and
    *locals* are given, they are used for the global and local variables,
    respectively.  If provided, *locals* can be any mapping object.  Remember
-   that at the module level, globals and locals are the same dictionary. If exec
-   gets two separate objects as *globals* and *locals*, the code will be
-   executed as if it were embedded in a class definition.
+   that at the module level, globals and locals are the same dictionary. If
+   ``exec`` gets two separate objects as *globals* and *locals*, the code will
+   be executed as if it were embedded in a class definition.
 
    If the *globals* dictionary does not contain a value for the key
    ``__builtins__``, a reference to the dictionary of the built-in module
@@ -631,7 +637,7 @@ are always available.  They are listed here in alphabetical order.
    .. note::
 
       The built-in functions :func:`globals` and :func:`locals` return the current
-      global and local dictionary, respectively, which may be useful to pass around
+      global and local namespace, respectively, which may be useful to pass around
       for use as the second and third argument to :func:`exec`.
 
    .. note::
@@ -647,6 +653,11 @@ are always available.  They are listed here in alphabetical order.
 
       The *globals* and *locals* arguments can now be passed as keywords.
 
+   .. versionchanged:: 3.13
+
+      The semantics of the default *locals* namespace have been adjusted as
+      described for the :func:`locals` builtin.
+
 
 .. function:: filter(function, iterable)
 
@@ -1063,10 +1074,10 @@ are always available.  They are listed here in alphabetical order.
 
     In all of the above cases, each call to ``locals()`` in a given frame of
     execution will return the *same* mapping object. Changes made through
-    the mapping object returned from ``locals()`` will be visible as bound,
-    rebound, or deleted local variables, and binding, rebinding, or deleting
-    local variables will immediately affect the contents of the returned mapping
-    object.
+    the mapping object returned from ``locals()`` will be visible as assigned,
+    reassigned, or deleted local variables, and assigning, reassigning, or
+    deleting local variables will immediately affect the contents of the
+    returned mapping object.
 
     At function scope (including for generators and coroutines), each call to
     ``locals()`` instead returns a fresh dictionary containing the current
@@ -1077,15 +1088,56 @@ are always available.  They are listed here in alphabetical order.
     cell references does *not* affect the contents of previously returned
     dictionaries.
 
+    Calling ``locals()`` as part of a comprehension in a function, generator, or
+    coroutine is equivalent to calling it in the containing scope, except that
+    the comprehension's initialised iteration variables will be included. In
+    other scopes, it behaves as if the comprehension were running as a nested
+    function.
+
+    Calling ``locals()`` as part of a generator expression is equivalent to
+    calling it in a nested generator function.
+
+
    .. versionchanged:: 3.13
-      In previous versions, the semantics of mutating the mapping object
-      returned from this function were formally undefined. In CPython
-      specifically, the mapping returned at function scope could be
-      implicitly refreshed by other operations, such as calling ``locals()``
-      again. Obtaining the legacy CPython behaviour now requires explicit
-      calls to update the initially returned dictionary with the results
-      of subsequent calls to ``locals()``.
+      As part of :pep:`667`, the semantics of mutating the mapping object
+      returned from this function are now formally defined. The behavior in
+      functions, generators, coroutines, comprehensions, and generator
+      expressions is now as described above. Aside from being defined, the
+      behaviour in other scopes remains unchanged from previous versions.
 
+   .. versionchanged:: 3.13
+      As part of :pep:`667`, the mapping returned when called in a function,
+      generator, coroutine, comprehension or generator expression is an
+      independent snapshot of the currently assigned local and locally
+      referenced nonlocal variables. In CPython specifically, the mapping
+      returned in these scopes could previously be implicitly refreshed by other
+      operations, such as calling ``locals()`` again. Obtaining the legacy
+      CPython behaviour now requires explicit calls to update the initially
+      returned dictionary with the results of subsequent calls to ``locals()``.
+
+   .. versionchanged:: 3.13
+      As part of :pep:`667`, calling `locals()` in a comprehension at module
+      or class scope (including via `exec` or `eval`) once more behaves as if
+      the comprehension were running as an independent nested function (i.e.
+      the local variables from the containing scope are not included).
+
+   .. versionchanged:: 3.12
+      As part of :pep:`709`, calling `locals()` in a comprehension at module
+      or class scope (including via `exec` or `eval`) produces an independent
+      snapshot of the containing scope with the comprehension iteration
+      variable(s) included.
+
+   .. versionchanged:: 3.12
+      As part of :pep:`709`, calling `locals()` in a comprehension as part of a
+      function, generator, or coroutine returns the same `locals()` reference
+      as calls in the containing scope.
+
+   ..
+      Possible simplification: change the 3.12 note to just "Changes to
+      behavior in comprehensions as described in :pep:`709`", and do somethine
+      similar for 3.13 and :pep:`667`. While the draft change notes are
+      comprehensive, they're also almost entirely in the "Who is really going
+      to care?" category.
 
 .. function:: map(function, iterable, *iterables)
 
@@ -1971,14 +2023,18 @@ are always available.  They are listed here in alphabetical order.
    :attr:`~object.__dict__` attributes (for example, classes use a
    :class:`types.MappingProxyType` to prevent direct dictionary updates).
 
-   Without an argument, :func:`vars` acts like :func:`locals`.  Note, the
-   locals dictionary is only useful for reads since updates to the locals
-   dictionary are ignored.
+   Without an argument, :func:`vars` acts like :func:`locals`.
 
    A :exc:`TypeError` exception is raised if an object is specified but
    it doesn't have a :attr:`~object.__dict__` attribute (for example, if
    its class defines the :attr:`~object.__slots__` attribute).
 
+   .. versionchanged:: 3.13
+
+      The semantics of calling this function without an argument have been
+      adjusted as described for the :func:`locals` builtin.
+
+
 .. function:: zip(*iterables, strict=False)
 
    Iterate over several iterables in parallel, producing tuples with an item

@@ -123,6 +123,11 @@ The typical usage to inspect a crashed program is::
    0
    (Pdb)
 
+.. versionchanged:: 3.13
+   The implementation of :pep:`667` means that name assignments made via `pdb`
+   will consistently and directly affect the active scope, even when running
+   inside a function, generator, or coroutine.
+
 
 The module defines the following functions; each enters the debugger in a
 slightly different way:
@@ -580,17 +585,9 @@ can be overridden by the local file.
 .. pdbcommand:: interact
 
    Start an interactive interpreter (using the :mod:`code` module) whose global
-   namespace contains all the (global and local) names found in the current
-   scope. Use ``exit()`` or ``quit()`` to exit the interpreter and return to
-   the debugger.
-
-   .. note::
-
-      Because interact creates a new global namespace with the current global
-      and local namespace for execution, assignment to variables will not
-      affect the original namespaces.
-      However, modification to the mutable objects will be reflected in the
-      original namespaces.
+   and local namespaces correspond to the global and local namespaces in the
+   current scope. Use ``exit()`` or ``quit()`` to exit the interpreter and
+   return to the debugger.
 
    .. versionadded:: 3.2
 
@@ -602,6 +599,11 @@ can be overridden by the local file.
       :pdbcmd:`interact` directs its output to the debugger's
       output channel rather than :data:`sys.stderr`.
 
+   .. versionchanged:: 3.13
+      The implementation of :pep:`667` means that name assignments made via the
+      interactive console will directly affect the active scope, even when
+      running inside a function, generator, or coroutine.
+
 .. _debugger-aliases:
 
 .. pdbcommand:: alias [name [command]]

diff --git a/Doc/library/profile.rst b/Doc/library/profile.rst
@@ -234,7 +234,7 @@ functions:
 .. function:: runctx(command, globals, locals, filename=None, sort=-1)
 
    This function is similar to :func:`run`, with added arguments to supply the
-   globals and locals dictionaries for the *command* string. This routine
+   globals and locals mappings for the *command* string. This routine
    executes::
 
       exec(command, globals, locals)

diff --git a/Doc/library/traceback.rst b/Doc/library/traceback.rst
@@ -473,7 +473,7 @@ in a :ref:`traceback <traceback-objects>`.
    attribute accessed (which also happens when casting it to a :class:`tuple`).
    :attr:`~FrameSummary.line` may be directly provided, and will prevent line
    lookups happening at all. *locals* is an optional local variable
-   dictionary, and if supplied the variable representations are stored in the
+   mapping, and if supplied the variable representations are stored in the
    summary for later display.
 
    :class:`!FrameSummary` instances have the following attributes:

diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst
@@ -94,10 +94,10 @@ Interpreter improvements:
   Performance improvements are modest -- we expect to be improving this
   over the next few releases.
 
-* :pep:`667`: :attr:`FrameType.f_locals <frame.f_locals>` when used in
-  a function now returns a write-through proxy to the frame's locals,
-  rather than a ``dict``. See the PEP for corresponding C API changes
-  and deprecations.
+* :pep:`667`: The `func:`locals` builtin now has defined semantics when mutating
+  the returned mapping. Python debuggers and similar tools may now more reliably
+  update local variables in optimised frames even during concurrent code
+  execution.
 
 New typing features:
 
@@ -370,6 +370,30 @@ disabled with the :envvar:`PYTHON_GIL` environment variable or the
 pip 24.1b1 or newer is required to install packages with C extensions in the
 free-threaded build.
 
+
+Defined mutation semantics for ``locals()``
+-------------------------------------------
+
+Historically, the semantics for mutating the return value of :func:`locals` have
+been left to individual Python implementations to define.
+
+Through :pep:`667`, Python 3.13 standardises the historical behaviour of CPython
+for most code execution scopes, but changes optimised scopes (functions,
+generators, coroutines, comprehensions, generator expressions) to explicitly
+return independent snapshots of the currently assigned local and nonlocal
+name references.
+
+To ensure debuggers and similar tools can reliably update local variables in
+affected scopes, :attr:`FrameType.f_locals <frame.f_locals>` now returns a
+write-through proxy to the frame's local and locally reference nonlocal
+variables rather than a ``dict`` with unclear runtime semantics.
+
+See :pep:`667` for more details, including related C API changes and
+deprecations.
+
+.. todo: Add notes to porting guide
+
+
 Other Language Changes
 ======================
 

diff --git a/Lib/code.py b/Lib/code.py
@@ -25,7 +25,7 @@ class InteractiveInterpreter:
     def __init__(self, locals=None):
         """Constructor.
 
-        The optional 'locals' argument specifies the dictionary in
+        The optional 'locals' argument specifies the mapping in
         which code will be executed; it defaults to a newly created
         dictionary with key "__name__" set to "__console__" and key
         "__doc__" set to None.