From 18f45b9cf4921aee9d40044965e21014fe6e4d98 Mon Sep 17 00:00:00 2001 From: Elvis Pranskevichus Date: Thu, 17 May 2018 16:43:39 -0400 Subject: [PATCH 1/2] bpo-32996: The bulk of What's New in Python 3.7 This represents the bulk of the content for What's New in Python 3.7. --- Doc/library/asyncio-task.rst | 2 +- Doc/library/ensurepip.rst | 3 - Doc/reference/compound_stmts.rst | 2 + Doc/whatsnew/3.7.rst | 2121 ++++++++++++++++++++++-------- 4 files changed, 1598 insertions(+), 530 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index db0e04a4a1d171..2488a909ec00fa 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -92,7 +92,7 @@ Coroutines (and tasks) can only run when the event loop is running. used in a callback-style code, wrap its result with :func:`ensure_future`. -.. function:: asyncio.run(coro, \*, debug=False) +.. function:: run(coro, \*, debug=False) This function runs the passed coroutine, taking care of managing the asyncio event loop and finalizing asynchronous diff --git a/Doc/library/ensurepip.rst b/Doc/library/ensurepip.rst index ed22180dc3f89f..c797f63326d1a2 100644 --- a/Doc/library/ensurepip.rst +++ b/Doc/library/ensurepip.rst @@ -78,9 +78,6 @@ options: Providing both of the script selection options will trigger an exception. -.. versionchanged:: 3.7.0 - The exit status is non-zero if the command fails. - Module API ---------- diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 153e85b3949175..5076e5df00789a 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -684,6 +684,8 @@ can be used to create instance variables with different implementation details. :pep:`3129` - Class Decorators +.. _async: + Coroutines ========== diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 24a5b3daf92afb..d9e73049495f88 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2,6 +2,8 @@ What's New In Python 3.7 **************************** +:Editor: Elvis Pranskevichus + .. Rules for maintenance: * Anyone can add text to this document. Do not spend very much time @@ -39,7 +41,7 @@ module. (Contributed by P.Y. Developer in :issue:`12345`.) - This saves the maintainer the effort of going through the Mercurial log + This saves the maintainer the effort of going through the Git log when researching a change. This article explains the new features in Python 3.7, compared to 3.6. @@ -53,20 +55,123 @@ For full details, see the :ref:`changelog `. so it's worth checking back even after reading earlier versions. -Summary -- Release highlights +Summary -- Release Highlights ============================= .. This section singles out the most important changes in Python 3.7. Brevity is key. +New syntax features: + +* :ref:`PEP 563 `, postponed evaluation of type annotations. + +Backwards incompatible syntax changes: + +* :keyword:`async` and :keyword:`await` are now reserved keywords. + +New library modules: + +* :mod:`contextvars`: :ref:`PEP 567 -- Context Variables ` +* :mod:`dataclasses`: :ref:`PEP 557 -- Data Classes ` +* :ref:`whatsnew37_importlib_resources` + +New built-in features: + +* :ref:`PEP 553 `, the new :func:`breakpoint` function. + +Python data model improvements: + +* :ref:`PEP 562 `, customization of access to + module attributes. + +* :ref:`PEP 560 `, core support for typing module and + generic types. -.. PEP-sized items next. +* the insertion-order preservation nature of :ref:`dict ` + objects `has been declared`_ to be an official + part of the Python language spec. +.. _has been declared: https://mail.python.org/pipermail/python-dev/2017-December/151283.html + +Significant improvements in the standard library: + +* The :mod:`asyncio` module has received new features, significant + :ref:`usability and performance improvements `. + +* The :mod:`time` module gained support for + :ref:`functions with nanosecond resolution `. + +CPython implementation improvements: + +* :ref:`PEP 552 `, deterministic .pycs +* :ref:`PEP 538 `, legacy C locale coercion +* :ref:`PEP 540 `, forced UTF-8 runtime mode +* :ref:`the new development runtime mode ` +* :ref:`PEP 565 `, improved :exc:`DeprecationWarning` + handling + +C API improvements: + +* :ref:`PEP 539 `, new C API for thread-local storage + +For a list of changes that may affect compatibility with previous Python +releases please refer to the :ref:`porting-to-python-37` section. New Features ============ +.. _whatsnew37-pep563: + +PEP 563: Postponed Evaluation of Annotations +-------------------------------------------- + +The advent of type hints in Python uncovered two glaring usability issues +with the functionality of annotations added in :pep:`3107` and refined +further in :pep:`526`: + +* annotations could only use names which were already available in the + current scope, in other words they didn't support forward references + of any kind; and + +* annotating source code had adverse effects on startup time of Python + programs. + +Both of these issues are fixed by postponing the evaluation of +annotations. Instead of compiling code which executes expressions in +annotations at their definition time, the compiler stores the annotation +in a string form equivalent to the AST of the expression in question. +If needed, annotations can be resolved at runtime using +:func:`typing.get_type_hints`. In the common case where this is not +required, the annotations are cheaper to store (since short strings +are interned by the interpreter) and make startup time faster. + +Usability-wise, annotations now support forward references, making the +following syntax valid:: + + class C: + @classmethod + def from_string(cls, source: str) -> C: + ... + + def validate_b(self, obj: B) -> bool: + ... + + class B: + ... + +Since this change breaks compatibility, the new behavior needs to be enabled +on a per-module basis in Python 3.7 using a :mod:`__future__` import:: + + from __future__ import annotations + +It will become the default in Python 4.0. + +.. seealso:: + + :pep:`563` -- Postponed evaluation of annotations + PEP written and implemented by Łukasz Langa. + .. _whatsnew37-pep538: @@ -106,19 +211,46 @@ legacy C locale remains active when the core interpreter is initialized. PEP written and implemented by Nick Coghlan. +.. _whatsnew37-pep540: + +PEP 540: Forced UTF-8 Runtime Mode +----------------------------------- + +The new :option:`-X` ``utf8`` command line option and :envvar:`PYTHONUTF8` +environment variable can be used to enable the CPython *UTF-8 mode*. + +When in UTF-8 mode, CPython ignores the locale settings, and uses the +UTF-8 encoding by default. The error handlers for :data:`sys.stdin` and +:data:`sys.stdout` streams are set to ``surrogateescape``. + +The forced UTF-8 mode can be used to change the text handling behavior in +an embedded Python interpreter without changing the locale settings of +an embedding application. + +The UTF-8 mode is enabled by default when the locale is "C". See +:ref:`whatsnew37-pep538` for details. + +.. seealso:: + + :pep:`540` -- Add a new UTF-8 mode + PEP written and implemented by Victor Stinner + + .. _whatsnew37-pep553: -PEP 553: Built-in breakpoint() ------------------------------- +PEP 553: Built-in ``breakpoint()`` +---------------------------------- -:pep:`553` describes a new built-in called :func:`breakpoint` which makes it -easy and consistent to enter the Python debugger. Built-in ``breakpoint()`` -calls :func:`sys.breakpointhook`. By default, this latter imports :mod:`pdb` and -then calls ``pdb.set_trace()``, but by binding ``sys.breakpointhook()`` to the -function of your choosing, ``breakpoint()`` can enter any debugger. Or, the -environment variable :envvar:`PYTHONBREAKPOINT` can be set to the callable of -your debugger of choice. Set ``PYTHONBREAKPOINT=0`` to completely disable -built-in ``breakpoint()``. +Python 3.7 includes the new built-in :func:`breakpoint` function as +an easy and consistent way to enter the Python debugger. + +Built-in ``breakpoint()`` calls :func:`sys.breakpointhook`. By default, this +latter imports :mod:`pdb` and then calls ``pdb.set_trace()``, but by binding +``sys.breakpointhook()`` to the function of your choosing, ``breakpoint()`` can +enter any debugger. Additionally, the environment variable +:envvar:`PYTHONBREAKPOINT` can be set to the callable of your debugger of +choice. Set ``PYTHONBREAKPOINT=0`` to completely disable built-in +``breakpoint()``. .. seealso:: @@ -128,8 +260,8 @@ built-in ``breakpoint()``. .. _whatsnew37-pep539: -PEP 539: A New C-API for Thread-Local Storage in CPython --------------------------------------------------------- +PEP 539: New C API for Thread-Local Storage +------------------------------------------- While Python provides a C API for thread-local storage support; the existing :ref:`Thread Local Storage (TLS) API ` has used @@ -158,17 +290,16 @@ effort will be made to add such support. PEP written by Erik M. Bray; implementation by Masayuki Yamamoto. -PEP 562: Customization of access to module attributes +.. _whatsnew37-pep562: + +PEP 562: Customization of Access to Module Attributes ----------------------------------------------------- -It is sometimes convenient to customize or otherwise have control over access -to module attributes. A typical example is managing deprecation warnings. -Typical workarounds are assigning :attr:`~instance.__class__` of a module -object to a custom subclass of :class:`types.ModuleType` or replacing the -:data:`sys.modules` item with a custom wrapper instance. This procedure is -now simplified by recognizing ``__getattr__`` defined directly in a module -that would act like a normal :meth:`__getattr__` method, except that it will -be defined on module *instances*. +Python 3.7 allows defining :meth:`__getattr__` on modules and will call +it whenever a module attribute is accessed. + +A typical example of where this may be useful is attribute access deprecation +warnings and lazy loading. .. seealso:: @@ -176,62 +307,13 @@ be defined on module *instances*. PEP written and implemented by Ivan Levkivskyi -PEP 563: Postponed evaluation of annotations --------------------------------------------- +.. _whatsnew37-pep564: -The advent of type hints in Python uncovered two glaring usability issues -with the functionality of annotations added in :pep:`3107` and refined -further in :pep:`526`: +PEP 564: New Time Functions With Nanosecond Resolution +------------------------------------------------------ -* annotations could only use names which were already available in the - current scope, in other words they didn't support forward references - of any kind; and - -* annotating source code had adverse effects on startup time of Python - programs. - -Both of these issues are fixed by postponing the evaluation of -annotations. Instead of compiling code which executes expressions in -annotations at their definition time, the compiler stores the annotation -in a string form equivalent to the AST of the expression in question. -If needed, annotations can be resolved at runtime using -``typing.get_type_hints()``. In the common case where this is not -required, the annotations are cheaper to store (since short strings -are interned by the interpreter) and make startup time faster. - -Usability-wise, annotations now support forward references, making the -following syntax valid:: - - class C: - @classmethod - def from_string(cls, source: str) -> C: - ... - - def validate_b(self, obj: B) -> bool: - ... - - class B: - ... - -Since this change breaks compatibility, the new behavior can be enabled -on a per-module basis in Python 3.7 using a :mod:`__future__` import, like -this:: - - from __future__ import annotations - -It will become the default in Python 4.0. - -.. seealso:: - - :pep:`563` -- Postponed evaluation of annotations - PEP written and implemented by Łukasz Langa. - - -PEP 564: Add new time functions with nanosecond resolution ----------------------------------------------------------- - -Add six new "nanosecond" variants of existing functions to the :mod:`time` -module: +:pep:`564` adds six new "nanosecond" variants of existing functions +to the :mod:`time` module: * :func:`time.clock_gettime_ns` * :func:`time.clock_settime_ns` @@ -240,12 +322,12 @@ module: * :func:`time.process_time_ns` * :func:`time.time_ns` -While similar to the existing functions without the ``_ns`` suffix, they -provide nanosecond resolution: they return a number of nanoseconds as a Python -``int``. +The new functions are similar in function to the existing functions +without the ``_ns`` suffix, but instead of returning fractional seconds, +they return nanoseconds as integers. -The ``time.time_ns()`` resolution is 3 times better than the ``time.time()`` -resolution on Linux and Windows. +On Linux and Windows the resolution of :func:`time.time_ns` is 3 times +better than that of :func:`time.time`. .. seealso:: @@ -291,73 +373,10 @@ by breaking changes in the APIs they used. PEP written and implemented by Nick Coghlan -PEP 540: Add a new UTF-8 mode ------------------------------ - -Add a new UTF-8 mode to ignore the locale, use the UTF-8 encoding, and change -:data:`sys.stdin` and :data:`sys.stdout` error handlers to ``surrogateescape``. -This mode is enabled by default in the POSIX locale, but otherwise disabled by -default. - -The new :option:`-X` ``utf8`` command line option and :envvar:`PYTHONUTF8` -environment variable are added to control the UTF-8 mode. - -.. seealso:: - - :pep:`540` -- Add a new UTF-8 mode - PEP written and implemented by Victor Stinner - - -.. _whatsnew37-pep557: - -PEP 557: Data Classes ---------------------- - -Adds a new module :mod:`dataclasses`. It provides a class decorator -:func:`~dataclasses.dataclass` which inspects the class's variable annotations (see -:pep:`526`) and using them, adds methods such as ``__init__``, -``__repr__``, and ``__eq__`` to the class. It is similar to -:class:`typing.NamedTuple`, but also works on classes with mutable -instances, among other features. - -For example:: - - @dataclass - class Point: - x: float - y: float - z: float = 0.0 - - p = Point(1.5, 2.5) - print(p) # produces "Point(x=1.5, y=2.5, z=0.0)" - -.. seealso:: +.. _whatsnew37-pep560: - :pep:`557` -- Data Classes - PEP written and implemented by Eric V. Smith - - -PEP 567: Context Variables --------------------------- - -Adds a new module :mod:`contextvars`, that provides APIs to manage, -store, and access non-local state. - -Context variables are natively supported in :mod:`asyncio` and are -ready to be used without any extra configuration. - -The :mod:`decimal` module was updated to use *contextvars* to store -the current decimal context. This allows decimal operations to work -with the correct context in async/await code. - -.. seealso:: - - :pep:`567` -- Context Variables - PEP written and implemented by Yury Selivanov - - -PEP 560: Core support for typing module and generic types ---------------------------------------------------------- +PEP 560: Core Support for ``typing`` module and Generic Types +------------------------------------------------------------- Initially :pep:`484` was designed in such way that it would not introduce *any* changes to the core CPython interpreter. Now type hints and the :mod:`typing` @@ -375,17 +394,23 @@ fixed. PEP written and implemented by Ivan Levkivskyi -New Development Mode: -X dev ----------------------------- +.. _whatsnew37-devmode: -Add a new "development mode": :option:`-X` ``dev`` command line option and -:envvar:`PYTHONDEVMODE` environment variable to enable CPython's "development -mode", introducing additional runtime checks which are too expensive to be -enabled by default. See :option:`-X` ``dev`` documentation for the effects of -the development mode. +Development Runtime Mode: -X dev +-------------------------------- -Hash-based pycs ---------------- +The new :option:`-X` ``dev`` command line option or the new +:envvar:`PYTHONDEVMODE` environment variable can be used to enable +CPython's *development mode*. When in development mode, CPython performs +additional runtime checks which are too expensive to be enabled by default. +See :option:`-X` ``dev`` documentation for the full description of the effects +of this mode. + + +.. _whatsnew37-pep552: + +PEP 552: Hash-based .pyc Files +------------------------------ Python has traditionally checked the up-to-dateness of bytecode cache files (i.e., ``.pyc`` files) by comparing the source metadata (last-modified timestamp @@ -422,6 +447,11 @@ Other Language Changes * :meth:`bytes.fromhex` and :meth:`bytearray.fromhex` now ignore all ASCII whitespace, not only spaces. (Contributed by Robert Xiao in :issue:`28927`.) +* :class:`str`, :class:`bytes`, and :class:`bytearray` gained support for + the new :meth:`isascii() ` method, which can be used to + test if a string or bytes contain only the ASCII characters. + (Contributed by INADA Naoki in :issue:`32677`.) + * :exc:`ImportError` now displays module name and module ``__file__`` path when ``from ... import ...`` fails. (Contributed by Matthias Bussonnier in :issue:`29546`.) @@ -446,35 +476,304 @@ Other Language Changes time when an import occurs) (Contributed by Nick Coghlan in :issue:`33053`.) +* The new :option:`-X` ``importtime`` option or the + :envvar:`PYTHONPROFILEIMPORTTIME` environment variable can be used to show + the timing of each module import. + (Contributed by Victor Stinner in :issue:`31415`.) + New Modules =========== +.. _whatsnew37-pep567: + +contextvars +----------- + +The new :mod:`contextvars` module and a set of new C APIs introduce +support for *context variables*. Context variables are conceptually +similar to thread-local variables. Unlike TLS, context variables +support asynchronous code correctly. + +The :mod:`asyncio` and :mod:`decimal` modules have been updated to use +and support context variables out of the box. Particularly the active +decimal context is now stored in a context variable, which allows +decimal operations to work with the correct context in asynchronous code. + +.. seealso:: + + :pep:`567` -- Context Variables + PEP written and implemented by Yury Selivanov + + +.. _whatsnew37-pep557: + +dataclasses +----------- + +The new :func:`~dataclasses.dataclass` decorator provides a way to declare +*data classes*. A data class describes its attributes using class variable +annotations. Its constructor and other magic methods, such as +:meth:`~object.__repr__`, :meth:`~object.__eq__`, and +:meth:`~object.__hash__` are generated automatically. + +Example:: + + @dataclass + class Point: + x: float + y: float + z: float = 0.0 + + p = Point(1.5, 2.5) + print(p) # produces "Point(x=1.5, y=2.5, z=0.0)" + +.. seealso:: + + :pep:`557` -- Data Classes + PEP written and implemented by Eric V. Smith + + +.. _whatsnew37_importlib_resources: + importlib.resources ------------------- -This module provides several new APIs and one new ABC for access to, opening, -and reading *resources* inside packages. Resources are roughly akin to files -inside of packages, but they needn't be actual files on the physical file -system. Module loaders can provide a :meth:`get_resource_reader()` function -which returns a :class:`importlib.abc.ResourceReader` instance to support this +The new :mod:`importlib.resources` module provides several new APIs and one +new ABC for access to, opening, and reading *resources* inside packages. +Resources are roughly similar to files inside packages, but they needn't +be actual files on the physical file system. Module loaders can provide a +:meth:`get_resource_reader()` function which returns +a :class:`importlib.abc.ResourceReader` instance to support this new API. Built-in file path loaders and zip file loaders both support this. -(see the PyPI package -`importlib_resources `_ -as a compatible back port for older Python versions). + +Contributed by Barry Warsaw and Brett Cannon in :issue:`32248`. + +.. seealso:: + + `importlib_resources `_ + -- a PyPI backport for earlier Python versions. Improved Modules ================ +abc +--- + +Improved conistency of exceptions raised by :func:`issubclass` when called +with a non-class and an abstract base class as the first and second +arguments, respectively. (Contributed by Josh Bronson in :issue:`33018`.) + + +aifc +---- + +Improved exceptions raised for invalid number of channels and sample +width when read an audio file in modules :mod:`aifc`, :mod:`wave` and +:mod:`sunau`. (Contributed by Serhiy Storchaka in :issue:`32056`.) + + argparse -------- -The :meth:`~argparse.ArgumentParser.parse_intermixed_args` supports letting -the user intermix options and positional arguments on the command line, -as is possible in many unix commands. It supports most but not all -argparse features. (Contributed by paul.j3 in :issue:`14191`.) +The new :meth:`ArgumentParser.parse_intermixed_args() \ +` method allows +intermixing options and positional arguments. +(Contributed by paul.j3 in :issue:`14191`.) + +:mod:`argparse` subparsers are now required by default. This matches the +behaviour in Python 2. To add an optional subparser, pass +``required=False`` to +:meth:`ArgumentParser.add_subparsers() `. +(Contributed by Anthony Sottile in :issue:`26510`.) + + +ast +--- + +:meth:`ast.literal_eval()` is now more strict. Addition and subtraction of +arbitrary numbers are no longer allowed. +(Contributed by Serhiy Storchaka in :issue:`31778`.) + + +.. _whatsnew37_asyncio: + +asyncio +------- + +The new :term:`provisional ` :func:`asyncio.run` function can +be used to run a coroutine from synchronous code by automatically creating and +destroying the event loop. +(Contributed by Yury Selivanov in :issue:`32314`.) + +The new :func:`asyncio.create_task` function has been added as a shortcut +to ``asyncio.get_event_loop().create_task()``. +(Contributed by Andrew Svetlov in :issue:`32311`.) + +The new :meth:`loop.start_tls() ` +method can be used to upgrade an existing connection to TLS. +(Contributed by Yury Selivanov in :issue:`23749`.) + +The new :meth:`loop.sock_recv_into() ` +method allows reading data from a socket directly into a provided buffer making +it possible to reduce data copies. +(Contributed by Antoine Pitrou in :issue:`31819`.) + +The new :func:`asyncio.current_task` function returns the currently running +:class:`~asyncio.Task` instance, and the new :func:`asyncio.all_tasks` +function returns a set of all existing ``Task`` instances in a given loop. +The :meth:`Task.current_task() ` and +:meth:`Task.all_tasks() ` methods have been deprecated. +(Contributed by Andrew Svetlov in :issue:`32250`.) + +The new *provisional* :class:`~asyncio.BufferedProtocol` class allows +implementing streaming protocols with manual control over the receive buffer. +(Contributed by Yury Selivanov in :issue:`32251`.) + +The new :func:`asyncio.get_running_loop` function returns the currently +running loop, or ``None`` if no loop is running. This is in contrast with +:func:`asyncio.get_event_loop`, which will *create* a new event loop if none +is running. +(Contributed by Yury Selivanov in :issue:`32269`.) + +The new :meth:`StreamWriter.wait_closed() ` +coroutine method allows waiting until the stream writer is closed. The new +:meth:`StreamWriter.is_closing() ` method +can be used to determine if the writer is closing. +(Contributed by Andrew Svetlov in :issue:`32391`.) + +The new :meth:`loop.sock_sendfile() ` +coroutine method allows sending files using :mod:`os.sendfile` when possible. +(Contributed by Andrew Svetlov in :issue:`32410`.) + +The new :meth:`Task.get_loop() ` and +:meth:`Future.get_loop() ` methods +return the instance of the loop on which a task or a future were created. +(Contributed by Yury Selivanov in :issue:`32415`.) +:meth:`Server.get_loop() ` allows to do the same for +:class:`asyncio.Server` objects. +(Contributed by Srinivas Reddy Thatiparthy in :issue:`32418`.) + +It is now possible to control how instances of :class:`asyncio.Server` begin +serving. Previously, the server would start serving immediately when created. +The new *start_serving* keyword argument to +:meth:`loop.create_server() ` and +:meth:`loop.create_unix_server() `, +as well as :meth:`Server.start_serving() `, and +:meth:`Server.serve_forever() ` +can be used to decouple server instantiaion and serving. The new +:meth:`Server.is_serving() ` method returns ``True`` +if the server is serving. :class:`~asyncio.Server` objects are now +asynchronous context managers:: + + srv = await loop.create_server(...) + + async with srv: + # some code + + # At this point, srv is closed and no longer accepts new connections. + +(Contributed by Yury Selivanov in :issue:`32662`.) + +Callback objects returned by +:func:`loop.call_later() ` +gained the new :meth:`when() ` method which +returns an absolute scheduled callback timestamp. +(Contributed by Andrew Svetlov in :issue:`32741`.) + +TCP sockets on Linux are now created with ``TCP_NODELAY`` flag set by default. +(Contributed by Yury Selivanov and Victor Stinner in :issue:`27456`.) + +The :meth:`loop.create_datagram_endpoint() \ +` method +gained support for Unix sockets. +(Contributed by Quentin Dawans in :issue:`31245`.) + +The :meth:`loop.create_connection() `, +:meth:`loop.create_server() `, +:meth:`loop.create_unix_server() `, and +:meth:`loop.create_accepted_socket() ` +now accept the *ssl_handshake_timeout* keyword argument. +(Contributed by Neil Aspinall in :issue:`29970`.) + +The new :meth:`Handle.cancelled() ` method returns +``True`` if the callback was cancelled. +(Contributed by Marat Sharafutdinov. in :issue:`31943`.) + +The :meth:`loop.create_unix_connection() ` +method now supports now supports passing a :term:`path-like objects ` +as a socket path. (Contributed by Yury Selivanov in :issue:`32066`.) + +The :func:`asyncio.windows_utils.socketpair`` function has been removed, +:func:`socket.socketpair` should be used instead. +(Contributed by Victor Stinner in :issue:`32154`.) + +The asyncio source has been converted to use the +:keyword:`async`/:keyword:`await` syntax. +(Contributed by Andrew Svetlov in :issue:`32193`.) + +Support for directly ``await``-ing instances of :class:`asyncio.Lock` and +other asyncio synchronization primitives has been deprecated. An +asynchronous context manager has to be used in order to acquire and release +the synchronization resource. See :ref:`async-with-locks` for more +information. +(Contributed by Andrew Svetlov in :issue:`32253`.) + +The :meth:`ReadTransport.resume_reading() ` +and :meth:`ReadTransport.pause_reading() ` +are now idempotent. +The new :meth:`ReadTransport.is_reading() ` +method can be used to determine the reading state of the transport. +(Contributed by Yury Selivanov in :issue:`32356`.) + +Exceptions occurring in cancelled tasks are no longer logged. +(Contributed by Yury Selivanov in :issue:`30508`.) + + +.. _whatsnew37-asyncio-optimizations: + +Optimizations +~~~~~~~~~~~~~ + +The :func:`asyncio.get_event_loop` function has been reimplemented in C to +make it up to 15 times faster. +(Contributed by Yury Selivanov in :issue:`32296`.) + +:class:`asyncio.Future` callback management has been optimized. +(Contributed by Yury Selivanov in :issue:`32348`.) + +:func:`asyncio.gather` is now up to 15% faster. +(Contributed by Yury Selivanov in :issue:`32355`.) + +:func:`asyncio.sleep` is now up to 2 times faster when the *delay* +argument is zero or negative. +(Contributed by Andrew Svetlov in :issue:`32351`.) + +The performance overhead of asyncio debug mode has been reduced. +(Contributed by Antoine Pitrou in :issue:`31970`.) + + +atexit +------ + +The :func:`~atexit.atexit` callbacks are now correctly tracked and called for +subinterpreters. +(Contributed by Marcel Plch in :issue:`31901`.) + +The traceback for :exc:`SystemExit` raised in an ``atexit`` callback +is no longer displayed. +(Contributed by Serhiy Storchaka in :issue:`28994`.) + + +bdb +--- + +Pdb and other debuggers dependent on :mod:`bdb` will correctly step over +(next command) native coroutines. +(Contributed by Pablo Galindo. in :issue:`32650`.) + binascii -------- @@ -483,6 +782,7 @@ The :func:`~binascii.b2a_uu` function now accepts an optional *backtick* keyword argument. When it's true, zeros are represented by ``'`'`` instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) + calendar -------- @@ -490,131 +790,450 @@ The class :class:`~calendar.HTMLCalendar` has new class attributes which ease the customisation of the CSS classes in the produced HTML calendar. (Contributed by Oz Tiram in :issue:`30095`.) +The :func:`~calendar.weekday` function now accepts any value for its *year* +argument (previously only values from ``1`` to ``9999`` were accepted). +(Contributed by Mark Gollahon in :issue:`28281`.) + +:meth:`Calendar.itermonthdates ` +will now consistently raise an exception when a date falls outside of the +``0001-01-01`` through ``9999-12-31`` range. To support applications that +cannot tolerate such exceptions, the new +:meth:`Calendar.itermonthdays3 ` and +:meth:`Calendar.itermonthdays4 ` can be used. +The new methods return tuples and are not restricted by the range supported by +:class:`datetime.date`. +(Contributed by Alexander Belopolsky in :issue:`28292`.) + + +collections +----------- + +The :func:`collections.namedtuple` factory has been reimplemented to rely +on precomputed methods to substantially reduce the use of :func:`exec`. +As a result, the *verbose* parameter and the *_source* attribute of the +named tuple are not longer supported. The benefits include: +1) having a smaller memory footprint for applications using multiple named +tuples, 2) faster creation of the named tuple class (approx 4x to 6x depending +on how it is measured), and 3) minor speed-ups for instance creation. +(Jelle Zijlstra with further improvements by INADA Naoki, Serhiy Storchaka, +and Raymond Hettinger in :issue:`28638`.) + +Additionally, ``collections.namedtuple()`` now supports default values. +(Contributed by Raymond Hettinger in :issue:`32320`.) + +:class:`collections.ChainMap` now preserves the order of the underlying +mappings. (Contributed by Raymond Hettinger in :issue:`32792`.) + + +compileall +---------- + +:func:`compileall.compile_dir` learned the new *invalidation_mode* parameter, +which can be used to enable +:ref:`hash-based .pyc invalidation `. The invalidation +mode can also be specified on the command line using the new +``--invalidation-mode`` argument. +(Contributed by Benjamin Peterson in :issue:`31650`.) + + +concurrent.futures +------------------ + +:class:`ProcessPoolExecutor ` and +:class:`ThreadPoolExecutor ` now +support the new *initializer* and *initargs* constructor arguments. +(Contributed by Antoine Pitrou in :issue:`21423`.) + +The :class:`ProcessPoolExecutor ` +can now take the multiprocessing context via the new *mp_context* argument. +(Contributed by Thomas Moreau in :issue:`31540`.) + +``executor.submit()`` now raises a :exc:`RuntimeError` if called during +interpreter shutdown. +(Contributed by Mark Nemec in :issue:`33097`.) + + +configparser +------------ + +The :class:`~configparser.ConfigParser`` constructor now uses ``read_dict()`` +to process the default values, making its behavior consistent with the +rest of the parser. Non-string keys and values in the defaults +dictionary are now being implicitly converted to strings. +(Contributed by James Tocknell. in :issue:`23835`.) + +The *filename* parameter of the +:meth:`ConfigParser.read() ` method now +accepts bytes objects. +(Contributed by Vincent Michel. in :issue:`31307`.) + + contextlib ---------- -:func:`~contextlib.asynccontextmanager` and -:class:`~contextlib.AbstractAsyncContextManager` have been added. (Contributed +The new :func:`~contextlib.nullcontext` is a simpler and faster alternative +to :class:`~contextlib.ExitStack` for handling simple context managers. +(Contributed by Jesse-Bakker in :issue:`10049`.) + +The new :func:`~contextlib.asynccontextmanager` and +:class:`~contextlib.AbstractAsyncContextManager` have been added to +complement their synchronous counterparts. (Contributed by Jelle Zijlstra in :issue:`29679` and :issue:`30241`.) -:class:`contextlib.AsyncExitStack` has been added. (Contributed by +The new :class:`contextlib.AsyncExitStack` class has been added as an +asynchronous equivalent to :class:`contextlib.ExitStack`. (Contributed by Alexander Mohr and Ilya Kulakov in :issue:`29302`.) + cProfile -------- :mod:`cProfile` command line now accepts ``-m module_name`` as an alternative to script path. (Contributed by Sanyam Khurana in :issue:`21862`.) + crypt ----- -Added support for the Blowfish method. +Added support for the Blowfish hashing method. (Contributed by Serhiy Storchaka in :issue:`31664`.) The :func:`~crypt.mksalt` function now allows to specify the number of rounds for hashing. (Contributed by Serhiy Storchaka in :issue:`31702`.) + +csv +--- + +A single empty field is now always quoted when written into a CSV +file. This allows to distinguish an empty row from a row consisting of +a single empty field. +(Contributed by Licht Takeuchi. in :issue:`32255`.) + + datetime -------- -Added the :meth:`datetime.fromisoformat ` +Added the :meth:`datetime.fromisoformat() ` method, which constructs a :class:`~datetime.datetime` object from a string in one of the formats output by -:meth:`datetime.isoformat `. +:meth:`datetime.isoformat() `. (Contributed by Paul Ganssle in :issue:`15873`.) +The :class:`tzinfo ` objects now supprt sub-minute offsets. +(Contributed by Alexander Belopolsky in :issue:`5288`.) + + +dbm +--- + +:func:`dbm.dumb` now supports reading read-only files and no longer writes the +index file when it is not changed. A deprecation warning is now emitted +if the index file is missing and recreated in the ``'r'`` and ``'w'`` +modes (this will be an error in future Python releases). +(Contributed by Serhiy Storchaka in :issue:`28847`.) + + +:func:`dbm.open` now encodes filenames using the filesystem encoding rather +than default encoding. +(Contributed by Serhiy Storchaka in :issue:`32922`.) + + +decimal +------- + +The :mod:`decimal` now uses :ref:`context variables ` to +store the decimal context. +(Contributed by Yury Selivanov in :issue:`32630`.) + + dis --- The :func:`~dis.dis` function now is able to disassemble nested code objects (the code of comprehensions, generator expressions and nested functions, and the code used for building nested -classes). (Contributed by Serhiy Storchaka in :issue:`11822`.) +classes). The maximum depth of disassembly recursion is controlled by +the new *depth* parameter. +(Contributed by Serhiy Storchaka in :issue:`11822`.) + +Functions in the :mod:`dis` module now works with asynchronous generators and +coroutine objects. +(Contributed by George Collins based on diagnosis by Luciano Ramalho +in :issue:`31183`.) + distutils --------- -``README.rst`` is now included in the list of distutils standard READMEs and -therefore included in source distributions. -(Contributed by Ryan Gonzalez in :issue:`11913`.) +``README.rst`` is now included in the list of distutils standard READMEs and +therefore included in source distributions. +(Contributed by Ryan Gonzalez in :issue:`11913`.) + +:class:`distutils.core.setup` now warns if the ``classifiers``, ``keywords`` +and ``platforms`` fields are not specified as a list or a string. +(Contributed by Berker Peksag in :issue:`19610`.) + +The ``upload`` command no longer tries to change CR end-of-line characters +to CRLF. This fixes a corruption issue with sdists that ended with a byte +equivalent to CR. +(Contributed by Bo Bayles in :issue:`32304`.) + +The unused ``install_misc`` command has been removed. +(Contributed by Eric N. Vander Weele. in :issue:`29218`.) + + +enum +---- + +The :class:`Enum ` learned the new ``_ignore_`` class property, +which allows listing the names of properties which should not become +enum members. +(Contributed by Ethan Furman in :issue:`31801`.) + +In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` +classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, +attempting to check for non-Flag objects in a :class:`Flag` member will +raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations +return :const:`False` instead and are deprecated. +(Contributed by Ethan Furman in :issue:`33217`.) + + +fpectl +------ + +The ``fpectl`` module has been removed. It was never enabled by +default, never worked correctly on x86-64, and it changed the Python +ABI in ways that caused unexpected breakage of C extensions. +(Contributed by Nathaniel J. Smith in :issue:`29137`.) + + +functools +--------- + +:func:`functools.singledispatch` now supports registering implementations +using type annotations. +(Contributed by Łukasz Langa in :issue:`32227`.) + + +gc +-- + +The new :func:`gc.freeze`` function allows freezing all objects tracked +by the garbage collector and exclude them from future collections. +This can be used before a POSIX ``fork()`` call to make the GC copy-on-write +friendly or to speed up collection. The new :func:`gc.unfreeze` functions +reverses this operation. Additionally, :func:`gc.get_freeze_count` can +be used to obtain the number of frozen objects. +(Contributed by Li Zekun in :issue:`31558`.) + -:class:`distutils.core.setup` now warns if the ``classifiers``, ``keywords`` -and ``platforms`` fields are not specified as a list or a string. -(Contributed by Berker Peksag in :issue:`19610`.) +hmac +---- + +The :mod:`hmac` module now has an optimized one-shot :func:`~hmac.digest` +function, which is up to three times faster than :func:`~hmac.HMAC`. +(Contributed by Christian Heimes in :issue:`32433`.) -The ``upload`` command no longer tries to change CR end-of-line characters -to CRLF. This fixes a corruption issue with sdists that ended with a byte -equivalent to CR. -(Contributed by Bo Bayles in :issue:`32304`.) http.client ----------- -Add configurable *blocksize* to :class:`~http.client.HTTPConnection` and -:class:`~http.client.HTTPSConnection` for improved upload throughput. +:class:`~http.client.HTTPConnection` and :class:`~http.client.HTTPSConnection` +now support the new *blocksize* argument for improved upload throughput. (Contributed by Nir Soffer in :issue:`31945`.) + http.server ----------- -:class:`~http.server.SimpleHTTPRequestHandler` supports the HTTP -``If-Modified-Since`` header. The server returns the 304 response status if the -target file was not modified after the time specified in the header. +:class:`~http.server.SimpleHTTPRequestHandler` now supports the HTTP +``If-Modified-Since`` header. The server returns the 304 response status if +the target file was not modified after the time specified in the header. (Contributed by Pierre Quentel in :issue:`29654`.) -Add the parameter *directory* to the :class:`~http.server.SimpleHTTPRequestHandler` -and the ``--directory`` to the command line of the module :mod:`http.server`. +:class:`~http.server.SimpleHTTPRequestHandler` accepts the new *directory* +argument, in addition to the new ``--directory`` command line argument. With this parameter, the server serves the specified directory, by default it uses the current working directory. (Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.) -hmac ----- +The new :class:`ThreadedHTTPServer ` class +uses threads to handle requests using :class:`~socketserver.ThreadingMixin`. +It is used when ``http.server`` is run with ``-m``. +(Contributed by Julien Palard in :issue:`31639`.) + + +idlelib and IDLE +---------------- + +IDLE module browser now shows nested classes and functions. +(Contributed by Guilherme Polo and Cheryl Sabella in :issue:`1612262`.) -The :mod:`hmac` module now has an optimized one-shot :func:`~hmac.digest` -function, which is up to three times faster than :func:`~hmac.HMAC`. -(Contributed by Christian Heimes in :issue:`32433`.) importlib --------- The :class:`importlib.abc.ResourceReader` ABC was introduced to -support the loading of resource from packages. +support the loading of resources from packages. See also +:ref:`whatsnew37_importlib_resources`. +(Contributed by Barry Warsaw, Brett Cannon in :issue:`32248`.) + +:func:`importlib.reload` now raises :exc:`ModuleNotFoundError` if the module +lacks a spec. +(Contributed by Garvit Khatri in :issue:`29851`.) + +:func:`importlib.find_spec` now raises ``ModuleNotFoundError`` instead of +:exc:`AttributeError` if the specified parent module is not a package (i.e. +lacks a ``__path__`` attribute). +(Contributed by Milan Oberkirch in :issue:`30436`.) + +The new :func:`importlib.source_hash` can be used to compute the hash of +the passed source. A :ref:`hash-based .pyc file ` +embeds the value returned by this function. + + +io +-- + +The new :meth:`TextIOWrapper.reconfigure() ` +method can be used to reconfigure the text stream with the new settings. +(Contributed by Antoine Pitrou in :issue:`30526` and +INADA Naoki in :issue:`15216`.) + + +ipaddress +--------- + +The new ``subnet_of`` and ``superset_of`` methods of +:class:`ipaddress.IPv6Network` and :class:`ipaddress.IPv4Network` can +be used for network containment tests. +(Contributed by Michel Albert and Cheryl Sabella. in :issue:`20825`.) + + +itertools +--------- + +:func:`itertools.islice` now accepts +:meth:`integer-like objects ` as start, stop, +and slice arguments. +(Contributed by Will Roberts in :issue:`30537`.) + locale ------ -Added another argument *monetary* in :func:`~locale.format_string` of :mod:`locale`. -If *monetary* is true, the conversion uses monetary thousands separator and -grouping strings. (Contributed by Garvit in :issue:`10379`.) +The new *monetary* argument to :func:`locale.format_string` can be used +to make the conversion use monetary thousands separators and +grouping strings. (Contributed by Garvit in :issue:`10379`.) + +The :func:`locale.format` function is deprecated. + +The :func:`locale.getpreferredencoding` function now always returns ``'UTF-8'`` +on Android or when in the :ref:`forced UTF-8 mode `. + + +logging +------- + +:class:`~logging.Logger` instances can now be pickled. +(Contributed by Vinay Sajip in :issue:`30520`.) + +The new :meth:`StreamHandler.setStream() `` +method can be used to replace the logger stream after handler creation. +(Contributed by Vinay Sajip in :issue:`30522`.) + +It is now possible to specify keyword arguments to handler constructor in +configuration passed to :func:`logging.config.fileConfig`. +(Contributed by Preston Landers in :issue:`31080`.) + + +macpath +------- + +The :mod:`macpath` is now deprecated and will be removed in Python +3.8. +(Contributed by Chi Hsuan Yen in :issue:`9850`.) -The :func:`~locale.getpreferredencoding` function now always returns ``'UTF-8'`` -on Android or in the UTF-8 mode (:option:`-X` ``utf8`` option), the locale and -the *do_setlocale* argument are ignored. math ---- -New :func:`~math.remainder` function, implementing the IEEE 754-style remainder -operation. (Contributed by Mark Dickinson in :issue:`29962`.) +The new :func:`math.remainder` function implements the IEEE 754-style remainder +operation. (Contributed by Mark Dickinson in :issue:`29962`.) + + +mimetypes +--------- + +The MIME type of .bmp has been changed from ``'image/x-ms-bmp'`` to +``'image/bmp'``. +(Contributed by Nitish Chandra in :issue:`22589`.) + + +msilib +------ + +The new :meth:`Database.Close() ` method can be used +to close the :abbr:`MSI` database. +(Contributed by Berker Peksag in :issue:`20486`.) + + +multiprocessing +--------------- + +The new :meth:`Process.close() ` method +explicitly closes the process object and releases all resources associated +with it. :exc:`ValueError` is raised if the underlying process is still +running. +(Contributed by Antoine Pitrou in :issue:`30596`.) + +The new :meth:`Process.kill() ` method can +be used to terminate the process using the :data:`SIGKILL` signal on Unix. +(Contributed by Vitor Pereira in :issue:`30794`.) + +Non-daemonic threads created by :class:`~multiprocessing.Process` are now +joined on process exit. +(Contributed by Antoine Pitrou in :issue:`18966`.) + os -- -Added support for :class:`bytes` paths in :func:`~os.fwalk`. (Contributed by -Serhiy Storchaka in :issue:`28682`.) +:func:`os.fwalk` now accepts the *path* argument as :class:`bytes`. +(Contributed by Serhiy Storchaka in :issue:`28682`.) -Added support for :ref:`file descriptors ` in :func:`~os.scandir` -on Unix. (Contributed by Serhiy Storchaka in :issue:`25996`.) +:func:`os.scandir` gained support for :ref:`file descriptors `. +(Contributed by Serhiy Storchaka in :issue:`25996`.) -New function :func:`~os.register_at_fork` allows registering Python callbacks -to be executed on a process fork. (Contributed by Antoine Pitrou in -:issue:`16500`.) +The new :func:`~os.register_at_fork` function allows registering Python +callbacks to be executed on a process fork. +(Contributed by Antoine Pitrou in :issue:`16500`.) + +Exposed the *preadv*, *preadv2*, *pwritev* and *pwritev2* system calls through +the new :func:`~os.preadv` and :func:`~os.pwritev` functions. +(Contributed by Pablo Galindo in :issue:`31368`.) + +The mode argument of :func:`os.makedirs` no longer affects the file +permission bits of newly-created intermediate-level directories. +(Contributed by Serhiy Storchaka in :issue:`19930`.) + +:func:`os.dup2` now returns the new file descriptor. Previously, ``None`` +was always returned. +(Contributed by Benjamin Peterson in :issue:`32441`.) + +The structure returned by :func:`os.stat` now contains the +:attr:`~os.stat_result.st_fstype` attribute on Solaris and its derivatives. +(Contributed by Jesús Cea Avión in :issue:`32659`.) + + +pathlib +------- + +The new :meth:`Path.is_mount() ` method is now available +on POSIX systems and can be used to determine whether a path is a mount point. +(Contributed by Cooper Ry Lees in :issue:`30897`.) -Exposed the system calls *preadv*, *preadv2*, *pwritev* and *pwritev2* through -the new functions :func:`~os.preadv` and :func:`~os.pwritev`. (Contributed by -Pablo Galindo in :issue:`31368`.) Exposed the system call *posix_spawn* through the new function :func:`~os.posix_spawn`. (Contributed by Pablo Galindo, Serhiy Storchaka and @@ -623,13 +1242,14 @@ Gregory P. Smith in :issue:`20104`.) pdb --- -:func:`~pdb.set_trace` now takes an optional *header* keyword-only -argument. If given, this is printed to the console just before debugging +:func:`pdb.set_trace` now takes an optional *header* keyword-only +argument. If given, it is printed to the console just before debugging begins. (Contributed by Barry Warsaw in :issue:`31389`.) :mod:`pdb` command line now accepts ``-m module_name`` as an alternative to script file. (Contributed by Mario Corchero in :issue:`32206`.) + py_compile ---------- @@ -641,6 +1261,22 @@ This allows for guaranteeing files when they are created eagerly. (Contributed by Bernhard M. Wiedemann in :issue:`29708`.) + +pydoc +----- + +The pydoc server can now bind to an arbitrary hostname specified by the +new ``-n`` command-line argument. +(Contributed by Feanil Patel in :issue:`31128`.) + + +queue +----- + +The new :class:`~queue.SimpleQueue` class is an unbounded :abbr:`FIFO` queue. +(Contributed by Antoine Pitrou in :issue:`14976`.) + + re -- @@ -652,70 +1288,127 @@ can be set within the scope of a group. ``'^$'`` or ``(?=-)`` that matches an empty string. (Contributed by Serhiy Storchaka in :issue:`25054`.) +Regular expressions compiled with the :const:`re.LOCALE` flag no longer +depend on the locale at compile time. Locale settings are applied only +when the compiled regular expression is used. +(Contributed by Serhiy Storchaka in :issue:`30215`.) + +:exc:`FutureWarning` is now emitted if a regular expression contains +character set constructs that will change semantically in the future, +such as nested sets and set operations. +(Contributed by Serhiy Storchaka in :issue:`30349`.) + +Compiled regular expression and match objects can now be copied +using :func:`copy.copy` and :func:`copy.deepcopy`. +(Contributed by Serhiy Storchaka in :issue:`10076`.) + + +signal +------ + +The new *warn_on_full_buffer* argument to the :func:`signal.set_wakeup_fd` +function makes it possible to specify whether Python prints a warning on +stderr when the wakeup buffer overflows. +(Contributed by Nathaniel J. Smith in :issue:`30050`.) + + +socket +------ + +The new :func:`socket.getblocking() ` method +returns ``True`` if the socket is in blocking mode and ``False`` otherwise. +(Contributed by Yury Selivanov in :issue:`32373`.) + +The new :func:`socket.close` function closes a passed socket file descriptor. +This function should be used instead of :func:`os.close` for better +compatibility across platforms. +(Contributed by Christian Heimes in :issue:`32454`.) + +The :mod:`socket` module now exposes the :data:`socket.TCP_CONGESTION` +(Linux 2.6.13), :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and +:data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. +(Contributed by Omar Sandoval. in :issue:`26273` and +Nathaniel J. Smith. in :issue:`29728`.) + +Support for :data:`socket.AF_VSOCK` sockets has been added to allow +communication between virtual machines and their hosts. +(Contributed by Cathy Avery in :issue:`27584`.) + +Sockets now auto-detect family, type and protocol from file descriptor +by default. +(Contributed by Christian Heimes in :issue:`28134`.) + sqlite3 ------- -:class:`sqlite3.Connection` now exposes a :class:`~sqlite3.Connection.backup` -method, if the underlying SQLite library is at version 3.6.11 or higher. +:class:`sqlite3.Connection` now exposes the :meth:`~sqlite3.Connection.backup` +method when the underlying SQLite library is at version 3.6.11 or higher. (Contributed by Lele Gaifax in :issue:`27645`.) +The *database* argument of :func:`sqlite3.connect` now accepts any +:term:`path-like object`, instead of just a string. +(Contributed by Anders Lorentsen in :issue:`31843`.) + ssl --- The :mod:`ssl` module now uses OpenSSL's builtin API instead of -:func:`~ssl.match_hostname` to check host name or IP address. Values -are validated during TLS handshake. Any cert validation error including -a failing host name match now raises :exc:`~ssl.SSLCertVerificationError` and -aborts the handshake with a proper TLS Alert message. The new exception -contains additional information. Host name validation can be customized -with :attr:`~ssl.SSLContext.host_flags`. +:func:`~ssl.match_hostname` to check a host name or an IP address. Values +are validated during TLS handshake. Any certificate validation error +including failing the host name check now raises +:exc:`~ssl.SSLCertVerificationError` and aborts the handshake with a proper +TLS Alert message. The new exception contains additional information. +Host name validation can be customized with +:attr:`SSLContext.host_flags `. (Contributed by Christian Heimes in :issue:`31399`.) .. note:: - The improved host name check requires an OpenSSL 1.0.2 or 1.1 compatible - libssl. OpenSSL 0.9.8 and 1.0.1 are no longer supported. LibreSSL is + The improved host name check requires OpenSSL 1.0.2 or 1.1 compatible + libssl. OpenSSL 0.9.8 and 1.0.1 are no longer supported. LibreSSL is temporarily not supported until it gains the necessary OpenSSL 1.0.2 APIs. -The ssl module no longer sends IP addresses in SNI TLS extension. +The ``ssl`` module no longer sends IP addresses in SNI TLS extension. (Contributed by Christian Heimes in :issue:`32185`.) :func:`~ssl.match_hostname` no longer supports partial wildcards like -``www*.example.org``. :attr:`~ssl.SSLContext.host_flags` has partial -wildcard matching disabled by default. +``www*.example.org``. :attr:`SSLContext.host_flags ` +has partial wildcard matching disabled by default. (Contributed by Mandeep Singh in :issue:`23033` and Christian Heimes in :issue:`31399`.) -The default cipher suite selection of the ssl module now uses a blacklist -approach rather than a hard-coded whitelist. Python no longer re-enables -ciphers that have been blocked by OpenSSL security update. Default cipher -suite selection can be configured on compile time. +The default cipher suite selection of the ``ssl`` module now uses a blacklist +approach rather than a hard-coded whitelist. Python no longer re-enables +ciphers that have been blocked by OpenSSL security updates. Default cipher +suite selection can be configured at compile time. (Contributed by Christian Heimes in :issue:`31429`.) -Added support for validating server certificates containing -internationalized domain names (IDNs). As part of this change, the -:attr:`ssl.SSLSocket.server_hostname` attribute now stores the -expected hostname in A-label form (``"xn--pythn-mua.org"``), rather -than the U-label form (``"pythön.org"``). (Contributed by +Validation of server certificates containing internationalized domain names +(IDNs) is now supported. As part of this change, the +:attr:`SSLSocket.server_hostname ` attribute +now stores the expected hostname in A-label form (``"xn--pythn-mua.org"``), +rather than the U-label form (``"pythön.org"``). (Contributed by Nathaniel J. Smith and Christian Heimes in :issue:`28414`.) -The ssl module has preliminary and experimental support for TLS 1.3 and -OpenSSL 1.1.1. (Contributed by Christian Heimes in :issue:`32947`, +The ``ssl`` module has preliminary and experimental support for TLS 1.3 and +OpenSSL 1.1.1. (Contributed by Christian Heimes in :issue:`32947`, :issue:`20995`, :issue:`29136`, and :issue:`30622`) :class:`~ssl.SSLSocket` and :class:`~ssl.SSLObject` no longer have a public -constructor. Direct instantiation was never a documented and supported -feature. Instances must be created with :class:`~ssl.SSLContext` methods +constructor. Direct instantiation was never a documented and supported +feature. Instances must be created with :class:`~ssl.SSLContext` methods :meth:`~ssl.SSLContext.wrap_socket` and :meth:`~ssl.SSLContext.wrap_bio`. (Contributed by Christian Heimes in :issue:`32951`) OpenSSL 1.1 APIs for setting the minimum and maximum TLS protocol version are -available as :attr:`~ssl.SSLContext.minimum_version` and -:attr:`~ssl.SSLContext.maximum_version`. Supported protocols are indicated -by new flags like :data:`~ssl.HAS_TLSv1_1`. +available as :attr:`SSLContext.minimum_version ` +and :attr:`SSLContext.maximum_version `. +Supported protocols are indicated by serveral new flags, such as +:data:`~ssl.HAS_TLSv1_1`. (Contributed by Christian Heimes in :issue:`32609`.) + string ------ @@ -723,32 +1416,61 @@ string expression pattern for braced placeholders and non-braced placeholders separately. (Contributed by Barry Warsaw in :issue:`1198569`.) + subprocess ---------- +The :func:`subprocess.run` function accepts the new *capture_output* +keyword argument. When ``True``, stdout and stderr will be captured. +This is equivalent to passing :data:`subprocess.PIPE` as *stdout* and +*stderr* arguments. +(Contributed by Bo Bayles in :issue:`32102`.) + +The ``subprocess.run`` function and the :class:`subprocess.Popen` constructor +now accept the *text* keyword argument as an alias +to *universal_newlines*. +(Contributed by Andrew Clegg in :issue:`31756`.) + On Windows the default for *close_fds* was changed from ``False`` to ``True`` when redirecting the standard handles. It's now possible to set *close_fds* to ``True`` when redirecting the standard handles. See -:class:`subprocess.Popen`. +:class:`subprocess.Popen`. This means that *close_fds* now defaults to +``True`` on all supported platforms. +(Contributed by Segev Finer in :issue:`19764`.) + +The subprocess module is now more graceful when handling +:exc:`KeyboardInterrupt` during :func:`subprocess.call`, +:func:`subprocess.run`, or in a :class:`~subprocess.Popen` +context manager. It now waits a short amount of time for the child +to exit, before continuing the handling of the ``KeyboardInterrupt`` +exception. +(Contributed by Gregory P. Smith in :issue:`25942`.) -This means that *close_fds* now defaults to ``True`` on all supported -platforms. (Contributed by Segev Finer in :issue:`19764`.) sys --- Added :attr:`sys.flags.dev_mode` flag for the new development mode. -tkinter -------- +The new :func:`sys.breakpointhook` hook function is called by the +built-in :func:`breakpoint`. +(Contributed by Barry Warsaw in :issue:`31353`.) + +On Android, the new :func:`sys.getandroidapilevel` returns the build-time +Android API version. +(Contributed by Victor Stinner in :issue:`28740`.) + +The new :func:`sys.get_coroutine_origin_tracking_depth` function returns +the current coroutine origin tracking depth, as set by +the new :func:`sys.set_coroutine_origin_tracking_depth`. +(Contributed by Nathaniel J. Smith in :issue:`32591`.) -Added :class:`tkinter.ttk.Spinbox`. -(Contributed by Alan Moore in :issue:`32585`.) time ---- -The :pep:`564` added six new functions with nanosecond resolution: +:pep:`564` adds six new functions with nanosecond resolution to the +:mod:`time` module: * :func:`time.clock_gettime_ns` * :func:`time.clock_settime_ns` @@ -757,7 +1479,7 @@ The :pep:`564` added six new functions with nanosecond resolution: * :func:`time.process_time_ns` * :func:`time.time_ns` -Add new clock identifiers: +New clock identifiers have been added: * :data:`time.CLOCK_BOOTTIME` (Linux): Identical to :data:`time.CLOCK_MONOTONIC`, except it also includes any time that the @@ -768,10 +1490,60 @@ Add new clock identifiers: the time the system has been running and not suspended, providing accurate uptime measurement, both absolute and interval. -Added functions :func:`time.thread_time` and :func:`time.thread_time_ns` -to get per-thread CPU time measurements. +The new :func:`time.thread_time` and :func:`time.thread_time_ns` functions +can be used to get per-thread CPU time measurements. (Contributed by Antoine Pitrou in :issue:`32025`.) +The new :func:`time.pthread_getcpuclockid` function returns the clock ID +of the thread-specific CPU-time clock. + + +timeit +------ + +:func:`timeit.timeit` autorange now uses a single loop iteration if +the benchmark takes less than 10 seconds, instead of 10 iterations. +``python3 -m timeit -s 'import time' 'time.sleep(1)'`` now takes 4 seconds +instead of 40 seconds. +(Contributed by Victor Stinner in :issue:`28240`.) + + +tkinter +------- + +The new :class:`tkinter.ttk.Spinbox` class is now available. +(Contributed by Alan Moore in :issue:`32585`.) + + +tracemalloc +----------- + +:class:`tracemalloc.Traceback` behaves more like regular tracebacks, +sorting the frames from oldest to most recent. +:meth:`Traceback.format() ` +now accepts negative *limit*, truncating the result to the +``abs(limit)`` oldest frames. To get the old behaviour, use +the new *most_recent_first* argument to ``Traceback.format()``. +(Contributed by Jesse Bakker. in :issue:`32121`.) + + +types +----- + +The new :class:`~types.WrapperDescriptorType`, +:class:`~types.MethodWrapperType`, and :class:`~types.MethodDescriptorType` +classes are now available. +(Contributed by Manuel Krebber and Guido van Rossum in :issue:`29377`.) + +The new :class:`~types.ClassMethodDescriptorType` is used to describe +class and static methods of builtin types. +(Contributed by Serhiy Storchaka in :issue:`32265`.) + +The new :func:`types.resolve_bases` function resolves MRO entries +dynamically as specified by :pep:`560`. +(Contributed by Ivan Levkivskyi in :issue:`32717`.) + + unicodedata ----------- @@ -779,6 +1551,7 @@ The internal :mod:`unicodedata` database has been upgraded to use `Unicode 10 `_. (Contributed by Benjamin Peterson.) + unittest -------- @@ -788,6 +1561,7 @@ tests ``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``, but not ``bar_tests.FooTest.test_something``. (Contributed by Jonas Haag in :issue:`32071`.) + unittest.mock ------------- @@ -800,6 +1574,7 @@ children by preventing to get or set any new attribute on the sealed mock. The sealing process is performed recursively. (Contributed by Mario Corchero in :issue:`30541`.) + urllib.parse ------------ @@ -807,6 +1582,7 @@ urllib.parse adding ``~`` to the set of characters that is never quoted by default. (Contributed by Christian Theune and Ratnadeep Debnath in :issue:`16285`.) + uu -- @@ -814,6 +1590,23 @@ Function :func:`~uu.encode` now accepts an optional *backtick* keyword argument. When it's true, zeros are represented by ``'`'`` instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) + +uuid +---- + +The new :attr:`UUID.is_safe ` attribute relays information +from the platform about whether generated UUIDs are generated with a +multiprocessing-safe method. +(Contributed by Barry Warsaw in :issue:`22807`.) + +:func:`uuid.getnode` now prefers universally administered +MAC addresses over locally administered MAC addresses. +This makes a better guarantee for global uniqueness of UUIDs returned +from :func:`uuid.uuid1`. If only locally administered MAC addresses are +available, the first such one found is returned. +(Contributed by Barry Warsaw in :issue:`32107`.) + + warnings -------- @@ -837,6 +1630,13 @@ The initialization of the default warnings filters has changed as follows: (Contributed by Nick Coghlan and Victor Stinner in :issue:`20361`, :issue:`32043`, and :issue:`32230`) +The default warning filter list now starts with a +``"default::DeprecationWarning:__main__"`` entry, so deprecation warnings +are once again shown by default in single-file scripts and at the +interactive prompt. See :ref:`whatsnew37-pep565` for details. +(Contributed by Nick Coghlan in :issue:`31975`.) + + xml.etree --------- @@ -845,13 +1645,15 @@ methods can now compare text of the current node with ``[. = "text"]``, not only text in children. Predicates also allow adding spaces for better readability. (Contributed by Stefan Behnel in :issue:`31648`.) + xmlrpc.server ------------- -:meth:`register_function` of :class:`~xmlrpc.server.SimpleXMLRPCDispatcher` and -its subclasses can be used as a decorator. (Contributed by Xiang Zhang in +:meth:`SimpleXMLRPCDispatcher.register_function ` +can now be used as a decorator. (Contributed by Xiang Zhang in :issue:`7769`.) + zipapp ------ @@ -865,143 +1667,245 @@ argument to generate a compressed archive. A command line option (Contributed by Zhiming Wang in :issue:`31638`.) +zipfile +------- + +:class:`~zipfile.ZipFile` now accepts the new *compresslevel* parameter to +control the compression level. +(Contributed by Stephen J. Turnbull. in :issue:`28102`.) + +Subdirectories in archives created by ``ZipFile`` are now stored in +alphabetical order. +(Contributed by Bernhard M. Wiedemann in :issue:`30693`.) + + +C API Changes +============= + +A new API for thread-local storage has been implemented. See +:ref:`whatsnew37-pep539` for an overview and +:ref:`thread-specific-storage-api` for a complete reference. +(Contributed by Masayuki Yamamoto in :issue:`25658`.) + +The new :c:func:`PyImport_GetModule` function returns the previously +imported module with the given name. +(Contributed by Eric Snow in :issue:`28411`.) + +The new :c:macro:`Py_RETURN_RICHCOMPARE` macro eases writing rich +comparison functions. +(Contributed by Petr Victorin in :issue:`23699`.) + +The new :c:macro:`Py_UNREACHABLE` macro can be used to mark unreachable +code paths. +(Contributed by Barry Warsaw in :issue:`31338`.) + +The :mod:`tracemalloc` now exposes a C API through the new +:c:func:`PyTraceMalloc_Track` and :c:func:`PyTraceMalloc_Untrack` +functions. +(Contributed by Victor Stinner in :issue:`30054`.) + +The new :c:func:`import__find__load__start` and +:c:func:`import__find__load__done` static markers can be used to trace +module imports. +(Contributed by Christian Heimes in :issue:`31574`.) + +The fields :c:member:`name` and :c:member:`doc` of structures +:c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, +:c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, +and :c:type:`wrapperbase` are now of type ``const char *`` rather of +``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) + +The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` +is now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy +Storchaka in :issue:`28769`.) + +The result of :c:func:`PyMapping_Keys`, :c:func:`PyMapping_Values` and +:c:func:`PyMapping_Items` is now always a list, rather than a list or a +tuple. (Contributed by Oren Milman in :issue:`28280`.) + +Added functions :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices`. +(Contributed by Serhiy Storchaka in :issue:`27867`.) + +:c:func:`PyOS_AfterFork` is deprecated in favour of the new functions +:c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` and +:c:func:`PyOS_AfterFork_Child`. (Contributed by Antoine Pitrou in +:issue:`16500`.) + +The ``PyExc_RecursionErrorInst`` singleton that was part of the public API +has been removed as its members being never cleared may cause a segfault +during finalization of the interpreter. Contributed by Xavier de Gaye in +:issue:`22898` and :issue:`30697`. + +Added C API support for timezones with timezone constructors +:c:func:`PyTimeZone_FromOffset` and :c:func:`PyTimeZone_FromOffsetAndName`, +and access to the UTC singleton with :c:data:`PyDateTime_TimeZone_UTC`. +Contributed by Paul Ganssle in :issue:`10381`. + +The type of results of :c:func:`PyThread_start_new_thread` and +:c:func:`PyThread_get_thread_ident`, and the *id* parameter of +:c:func:`PyThreadState_SetAsyncExc` changed from :c:type:`long` to +:c:type:`unsigned long`. +(Contributed by Serhiy Storchaka in :issue:`6532`.) + +:c:func:`PyUnicode_AsWideCharString` now raises a :exc:`ValueError` if the +second argument is *NULL* and the :c:type:`wchar_t*` string contains null +characters. (Contributed by Serhiy Storchaka in :issue:`30708`.) + +Changes to the startup sequence and the management of dynamic memory +allocators mean that the long documented requirement to call +:c:func:`Py_Initialize` before calling most C API functions is now +relied on more heavily, and failing to abide by it may lead to segfaults in +embedding applications. See the :ref:`porting-to-python-37` section in this +document and the :ref:`pre-init-safe` section in the C API documentation +for more details. + +The new :c:func:`PyInterpreterState_GetID` returns the unique ID for a +given interpreter. +(Contributed by Eric Snow in :issue:`29102`.) + +:c:func:`Py_DecodeLocale`, :c:func:`Py_EncodeLocale` now use the UTF-8 +encoding when the :ref:`UTF-8 mode ` is enabled. +(Contributed by Victor Stinner in :issue:`29240`.) + +:c:func:`PyUnicode_DecodeLocaleAndSize` and :c:func:`PyUnicode_EncodeLocale` +now use the current locale encoding for ``surrogateescape`` error handler. +(Contributed by Victor Stinner in :issue:`29240`.) + +The *start* and *end* parameters of :c:func:`PyUnicode_FindChar` are +now adjusted to behave like ``str[start:end]``. +(Contributed by Xiang Zhang in :issue:`28822`.) + + +Build Changes +============= + +Support for building ``--without-threads`` has been removed. The +:mod:`threading` module is now always available. +(Contributed by Antoine Pitrou in :issue:`31370`.). + +A full copy of libffi is no longer bundled for use when building the +:mod:`_ctypes ` module on non-OSX UNIX platforms. An installed copy +of libffi is now required when building ``_ctypes`` on such platforms. +(Contributed by Zachary Ware in :issue:`27979`.) + +The Windows build process no longer depends on Subversion to pull in external +sources, a Python script is used to download zipfiles from GitHub instead. +If Python 3.6 is not found on the system (via ``py -3.6``), NuGet is used to +download a copy of 32-bit Python for this purpose. (Contributed by Zachary +Ware in :issue:`30450`.) + +The :mod:`ssl` module requires OpenSSL 1.0.2 or 1.1 compatible libssl. +OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer +supported. LibreSSL is temporarily not supported as well. LibreSSL releases +up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs. + + Optimizations ============= -* Added two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD` to avoid - instantiation of bound method objects for method calls, which results - in method calls being faster up to 20%. (Contributed by Yury Selivanov and - INADA Naoki in :issue:`26110`.) - -* Searching some unlucky Unicode characters (like Ukrainian capital "Є") - in a string was up to 25 times slower than searching other characters. - Now it is slower only by 3 times in the worst case. - (Contributed by Serhiy Storchaka in :issue:`24821`.) - -* Fast implementation from standard C library is now used for functions - :func:`~math.erf` and :func:`~math.erfc` in the :mod:`math` module. - (Contributed by Serhiy Storchaka in :issue:`26121`.) - -* The :func:`os.fwalk` function has been sped up by 2 times. This was done - using the :func:`os.scandir` function. - (Contributed by Serhiy Storchaka in :issue:`25996`.) - -* The :func:`shutil.rmtree` function has been sped up to 20--40%. - This was done using the :func:`os.scandir` function. - (Contributed by Serhiy Storchaka in :issue:`28564`.) - -* Optimized case-insensitive matching and searching of :mod:`regular - expressions `. Searching some patterns can now be up to 20 times faster. - (Contributed by Serhiy Storchaka in :issue:`30285`.) - -* :func:`re.compile` now converts ``flags`` parameter to int object if - it is ``RegexFlag``. It is now as fast as Python 3.5, and faster than - Python 3.6 by about 10% depending on the pattern. - (Contributed by INADA Naoki in :issue:`31671`.) - -* :meth:`~selectors.BaseSelector.modify` methods of classes - :class:`selectors.EpollSelector`, :class:`selectors.PollSelector` - and :class:`selectors.DevpollSelector` may be around 10% faster under - heavy loads. (Contributed by Giampaolo Rodola' in :issue:`30014`) - -* Constant folding is moved from peephole optimizer to new AST optimizer. - (Contributed by Eugene Toder and INADA Naoki in :issue:`29469`) - -* Most functions and methods in :mod:`abc` have been rewritten in C. - This makes creation of abstract base classes, and calling :func:`isinstance` - and :func:`issubclass` on them 1.5x faster. This also reduces Python - start-up time by up to 10%. (Contributed by Ivan Levkivskyi and INADA Naoki - in :issue:`31333`) - -* Significant speed improvements to alternate constructors for - :class:`datetime.date` and :class:`datetime.datetime` by using fast-path - constructors when not constructing subclasses. (Contributed by Paul Ganssle - in :issue:`32403`) - - -Build and C API Changes -======================= - -* :mod:`py_compile` and :mod:`compileall` now support the - :envvar:`SOURCE_DATE_EPOCH` environment variable by unconditionally - building ``.pyc`` files for hash verification instead of potentially - timestamp-based ``.pyc`` files. See the notes for the `py_compile`_ - improvement notes for more details. - -* A full copy of libffi is no longer bundled for use when building the - :mod:`_ctypes ` module on non-OSX UNIX platforms. An installed copy - of libffi is now required when building ``_ctypes`` on such platforms. - (Contributed by Zachary Ware in :issue:`27979`.) - -* The fields :c:member:`name` and :c:member:`doc` of structures - :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, - :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, - and :c:type:`wrapperbase` are now of type ``const char *`` rather of - ``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) - -* The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` - is now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy - Storchaka in :issue:`28769`.) - -* The result of :c:func:`PyMapping_Keys`, :c:func:`PyMapping_Values` and - :c:func:`PyMapping_Items` is now always a list, rather than a list or a - tuple. (Contributed by Oren Milman in :issue:`28280`.) - -* Added functions :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices`. - (Contributed by Serhiy Storchaka in :issue:`27867`.) - -* :c:func:`PyOS_AfterFork` is deprecated in favour of the new functions - :c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` and - :c:func:`PyOS_AfterFork_Child`. (Contributed by Antoine Pitrou in - :issue:`16500`.) - -* The Windows build process no longer depends on Subversion to pull in external - sources, a Python script is used to download zipfiles from GitHub instead. - If Python 3.6 is not found on the system (via ``py -3.6``), NuGet is used to - download a copy of 32-bit Python for this purpose. (Contributed by Zachary - Ware in :issue:`30450`.) - -* The ``PyExc_RecursionErrorInst`` singleton that was part of the public API - has been removed as its members being never cleared may cause a segfault - during finalization of the interpreter. Contributed by Xavier de Gaye in - :issue:`22898` and :issue:`30697`. - -* Support for building ``--without-threads`` is removed. - (Contributed by Antoine Pitrou in :issue:`31370`.). - -* Added C API support for timezones with timezone constructors - :c:func:`PyTimeZone_FromOffset` and :c:func:`PyTimeZone_FromOffsetAndName`, - and access to the UTC singleton with :c:data:`PyDateTime_TimeZone_UTC`. - Contributed by Paul Ganssle in :issue:`10381`. - -- The type of results of :c:func:`PyThread_start_new_thread` and - :c:func:`PyThread_get_thread_ident`, and the *id* parameter of - :c:func:`PyThreadState_SetAsyncExc` changed from :c:type:`long` to - :c:type:`unsigned long`. - (Contributed by Serhiy Storchaka in :issue:`6532`.) - -- :c:func:`PyUnicode_AsWideCharString` now raises a :exc:`ValueError` if the - second argument is *NULL* and the :c:type:`wchar_t*` string contains null - characters. (Contributed by Serhiy Storchaka in :issue:`30708`.) - -- Changes to the startup sequence and the management of dynamic memory - allocators mean that the long documented requirement to call - :c:func:`Py_Initialize` before calling most C API functions is now - relied on more heavily, and failing to abide by it may lead to segfaults in - embedding applications. See the :ref:`porting-to-python-37` section in this - document and the :ref:`pre-init-safe` section in the C API documentation - for more details. +The overhead of calling many methods of various standard library classes +implemented in C has been significantly reduced by porting more code +to use the ``METH_FASTCALL`` convention. +(Contributed by Victor Stinner in :issue:`29300`, :issue:`29507`, +:issue:`29452`, and :issue:`29286`.) + +Various optimizations have reduced Python startup time by 5% on Linux and +up to 30% on macOS. +(Contributed by Victor Stinner, INADA Naoki in :issue:`29585`.) + +Method calls are now up to 20% faster due to the bytecode changes which +avoid creating bound method instances. +(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) + +The :mod:`asyncio` received a number of notable optimizations for commonly +used functions. +See :ref:`optimizations in asyncio ` for +details. + +:func:`sorted` and :meth:`list.sort` have been optimized for common cases +to be up to 40-75% faster. +(Contributed by Elliot Gorokhovsky in :issue:`28685`.) + +:meth:`dict.copy` is now up to 5.5 times faster. +(Contributed by Yury Selivanov in :issue:`31179`.) + +:func:`hasattr` and :func:`getattr`` are now about 4 times faster when +*name* is not found and *obj* does not override :meth:`object.__getattr__` +or :meth:`object.__getattribute__`. +(Contributed by INADA Naoki in :issue:`32544`.) + +Searching for certain Unicode characters (like Ukrainian capital "Є") +in a string was up to 25 times slower than searching for other characters. +It is now only 3 times slower in the worst case. +(Contributed by Serhiy Storchaka in :issue:`24821`.) + +The :func:`collections.namedtuple` factory has been reimplemented to +make the creation of named tuples 4 to 6 times faster. +(Contributed by Jelle Zijlstra with further improvements by INADA Naoki, +Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) + +:meth:`date.fromordinal` and :meth:`date.fromtimestamp` are now up to +30% faster in the common case. +(Contributed by Paul Ganssle in :issue:`32403`.) + +Fast implementation from standard C library is now used for the +:func:`math.erf` and :func:`math.erfc` functions. +(Contributed by Serhiy Storchaka in :issue:`26121`.) + +The :func:`os.fwalk` function is now up to 2 times faster thanks to +the use of :func:`os.scandir`. +(Contributed by Serhiy Storchaka in :issue:`25996`.) + +The speed of the :func:`shutil.rmtree` function has been improved by +20--40% thanks to the use of the :func:`os.scandir` function. +(Contributed by Serhiy Storchaka in :issue:`28564`.) + +Optimized case-insensitive matching and searching of :mod:`regular +expressions `. Searching some patterns can now be up to 20 times faster. +(Contributed by Serhiy Storchaka in :issue:`30285`.) + +:func:`re.compile` now converts ``flags`` parameter to int object if +it is ``RegexFlag``. It is now as fast as Python 3.5, and faster than +Python 3.6 by about 10% depending on the pattern. +(Contributed by INADA Naoki in :issue:`31671`.) + +The :meth:`~selectors.BaseSelector.modify` methods of classes +:class:`selectors.EpollSelector`, :class:`selectors.PollSelector` +and :class:`selectors.DevpollSelector` may be around 10% faster under +heavy loads. (Contributed by Giampaolo Rodola' in :issue:`30014`) + +Constant folding has been moved from the peephole optimizer to the new AST +optimizer, which is able perform optimizations more consistently. +(Contributed by Eugene Toder and INADA Naoki in :issue:`29469` and +:issue:`11549`.) + +Most functions and methods in :mod:`abc` have been rewritten in C. +This makes creation of abstract base classes, and calling :func:`isinstance` +and :func:`issubclass` on them 1.5x faster. This also reduces Python +start-up time by up to 10%. (Contributed by Ivan Levkivskyi and INADA Naoki +in :issue:`31333`) + +Significant speed improvements to alternate constructors for +:class:`datetime.date` and :class:`datetime.datetime` by using fast-path +constructors when not constructing subclasses. (Contributed by Paul Ganssle +in :issue:`32403`) + +The speed of comparison of :class:`array.array` instances has been +improved considerably in certain cases. It is now from 10x to 70x faster +when comparing arrays holding values of the same integer type. +(Contributed by Adrian Wielgosik in :issue:`24700`.) + +The :func:`math.erf` and :func:`math.erfc` functions now use the (faster) +C library implementation on most platforms. +(Contributed by Serhiy Storchaka in :issue:`26121`.) Other CPython Implementation Changes ==================================== -* Trace hooks may now opt out of receiving ``line`` events from the interpreter - by setting the new ``f_trace_lines`` attribute to :const:`False` on the frame - being traced. (Contributed by Nick Coghlan in :issue:`31344`.) - -* Trace hooks may now opt in to receiving ``opcode`` events from the interpreter - by setting the new ``f_trace_opcodes`` attribute to :const:`True` on the frame +* Trace hooks may now opt out of receiving the ``line`` and ``opcode`` events + from the interpreter by setting the corresponding new ``f_trace_lines`` and + ``f_trace_opcodes`` attributes to :const:`False` on the frame being traced. (Contributed by Nick Coghlan in :issue:`31344`.) * Fixed some consistency problems with namespace package module attributes. @@ -1012,90 +1916,193 @@ Other CPython Implementation Changes ``__loader__`` (previously, the former was set to ``None``). See :issue:`32303`. +* The :func:`locals` dictionary now displays in the lexical order that + variables were defined. Previously, the order was reversed. + (Contributed by Raymond Hettinger in :issue:`32690`.) -Deprecated -========== -* In Python 3.8, the abstract base classes in :mod:`collections.abc` will no - longer be exposed in the regular :mod:`collections` module. This will help - create a clearer distinction between the concrete classes and the abstract - base classes. +Deprecated Python Behavior +========================== -* Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated - in comprehensions and generator expressions (aside from the iterable expression - in the leftmost :keyword:`for` clause). This ensures that comprehensions - always immediately return a container of the appropriate type (rather than - potentially returning a :term:`generator iterator` object), while generator - expressions won't attempt to interleave their implicit output with the output - from any explicit yield expressions. +Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated +in comprehensions and generator expressions (aside from the iterable expression +in the leftmost :keyword:`for` clause). This ensures that comprehensions +always immediately return a container of the appropriate type (rather than +potentially returning a :term:`generator iterator` object), while generator +expressions won't attempt to interleave their implicit output with the output +from any explicit yield expressions. In Python 3.7, such expressions emit +:exc:`DeprecationWarning` when compiled, in Python 3.8 this will be a +:exc:`SyntaxError`. +(Contributed by Serhiy Storchaka in :issue:`10544`.) - In Python 3.7, such expressions emit :exc:`DeprecationWarning` when compiled, - in Python 3.8+ they will emit :exc:`SyntaxError`. (Contributed by Serhiy - Storchaka in :issue:`10544`.) +Returning a subclass of :class:`complex` from :meth:`object.__complex__` is +deprecated and will be an error in future Python versions. This makes +``__complex__()`` consistent with :meth:`object.__int__` and +:meth:`object.__float__`. +(Contributed by Serhiy Storchaka in :issue:`28894`.) -- Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with - a macro if ``Py_LIMITED_API`` is not set or set to the value between - ``0x03050400`` and ``0x03060000`` (not including) or ``0x03060100`` or - higher. (Contributed by Serhiy Storchaka in :issue:`27867`.) -- Deprecated :meth:`format` from :mod:`locale`, use the :meth:`format_string` - instead. (Contributed by Garvit in :issue:`10379`.) -- Methods - :meth:`MetaPathFinder.find_module() ` - (replaced by - :meth:`MetaPathFinder.find_spec() `) - and - :meth:`PathEntryFinder.find_loader() ` - (replaced by - :meth:`PathEntryFinder.find_spec() `) - both deprecated in Python 3.4 now emit :exc:`DeprecationWarning`. - (Contributed by Matthias Bussonnier in :issue:`29576`) +Deprecated Python modules, functions and methods +================================================ -- Using non-integer value for selecting a plural form in :mod:`gettext` is - now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka - in :issue:`28692`.) +aifc +---- -- The :mod:`macpath` is now deprecated and will be removed in Python 3.8. +:func:`aifc.openfp` has been deprecated and will be removed in Python 3.9. +Use :func:`aifc.open` instead. +(Contributed by Brian Curtin in :issue:`31985`.) -- The :class:`importlib.abc.ResourceLoader` ABC has been deprecated in - favour of :class:`importlib.abc.ResourceReader`. -- Deprecated :func:`sys.set_coroutine_wrapper` and - :func:`sys.get_coroutine_wrapper`. +asyncio +------- -- :func:`ssl.wrap_socket` is deprecated. Use - :meth:`ssl.SSLContext.wrap_socket` instead. - (Contributed by Christian Heimes in :issue:`28124`.) +Support for directly ``await``-ing instances of :class:`asyncio.Lock` and +other asyncio synchronization primitives has been deprecated. An +asynchronous context manager has to be used in order to acquire and release +the synchronization resource. See :ref:`async-with-locks` for more +information. +(Contributed by Andrew Svetlov in :issue:`32253`.) +The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks` +methods have been deprecated. +(Contributed by Andrew Svetlov in :issue:`32250`.) + + +collections +----------- + +In Python 3.8, the abstract base classes in :mod:`collections.abc` will no +longer be exposed in the regular :mod:`collections` module. This will help +create a clearer distinction between the concrete classes and the abstract +base classes. +(Contributed by Serhiy Storchaka in :issue:`25988`.) + + +enum +---- + +In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` +classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, +attempting to check for non-Flag objects in a :class:`Flag` member will +raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations +return :const:`False` instead. +(Contributed by Ethan Furman in :issue:`33217`.) + + +gettext +------- + +Using non-integer value for selecting a plural form in :mod:`gettext` is +now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka +in :issue:`28692`.) -Windows Only ------------- -- The python launcher, (py.exe), can accept 32 & 64 bit specifiers **without** - having to specify a minor version as well. So ``py -3-32`` and ``py -3-64`` - become valid as well as ``py -3.7-32``, also the -*m*-64 and -*m.n*-64 forms - are now accepted to force 64 bit python even if 32 bit would have otherwise - been used. If the specified version is not available py.exe will error exit. - (Contributed by Steve Barnes in :issue:`30291`.) +importlib +--------- + +Methods +:meth:`MetaPathFinder.find_module() ` +(replaced by +:meth:`MetaPathFinder.find_spec() `) +and +:meth:`PathEntryFinder.find_loader() ` +(replaced by +:meth:`PathEntryFinder.find_spec() `) +both deprecated in Python 3.4 now emit :exc:`DeprecationWarning`. +(Contributed by Matthias Bussonnier in :issue:`29576`) + +The :class:`importlib.abc.ResourceLoader` ABC has been deprecated in +favour of :class:`importlib.abc.ResourceReader`. + + +locale +------ + +:func:`locale.format` has been deprecated, use :meth:`locale.format_string` +instead. (Contributed by Garvit in :issue:`10379`.) + + +macpath +------- + +The :mod:`macpath` is now deprecated and will be removed in Python 3.8. + + +threading +--------- + +:mod:`dummy_threading` and :mod:`_dummy_thread` have been deprecated. It is +no longer possible to build Python with threading disabled. +Use :mod:`threading` instead. +(Contributed by Antoine Pitrou in :issue:`31370`.) + + +socket +------ + +The silent argument value trunctation in :func:`socket.htons` and +:func:`socket.ntohs` has been deprecated. In future versions of Python, +if the passed argument is larger than 16 bits, an exception will be raised. +(Contributed by Oren Milman in :issue:`28332`.) + + +ssl +--- + +:func:`ssl.wrap_socket` is deprecated. Use +:meth:`ssl.SSLContext.wrap_socket` instead. +(Contributed by Christian Heimes in :issue:`28124`.) + + +sunau +----- + +:func:`sunau.openfp` has been deprecated and will be removed in Python 3.9. +Use :func:`sunau.open` instead. +(Contributed by Brian Curtin in :issue:`31985`.) + + +sys +--- + +Deprecated :func:`sys.set_coroutine_wrapper` and +:func:`sys.get_coroutine_wrapper`. + +The undocumented ``sys.callstats()`` function has been deprecated and +will be removed in a future Python version. +(Contributed by Victor Stinner in :issue:`28799`.) + + +wave +---- + +:func:`wave.openfp` has been deprecated and will be removed in Python 3.9. +Use :func:`wave.open` instead. +(Contributed by Brian Curtin in :issue:`31985`.) -- The launcher can be run as ``py -0`` to produce a list of the installed pythons, - *with default marked with an asterisk*. Running ``py -0p`` will include the paths. - If py is run with a version specifier that cannot be matched it will also print - the *short form* list of available specifiers. - (Contributed by Steve Barnes in :issue:`30362`.) +Deprecated functions and types of the C API +=========================================== + +Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with +a macro if ``Py_LIMITED_API`` is not set or set to the value between +``0x03050400`` and ``0x03060000`` (not including) or ``0x03060100`` or +higher. (Contributed by Serhiy Storchaka in :issue:`27867`.) + +:c:func:`PyOS_AfterFork` has been deprecated. Use :c:func:`PyOS_BeforeFork`, +:c:func:`PyOS_AfterFork_Parent` or :c:func:`PyOS_AfterFork_Child()` instead. +(Contributed by Antoine Pitrou in :issue:`16500`.) -Removed -======= Platform Support Removals -------------------------- +========================= + +FreeBSD 9 and older are no longer officially supported. -* FreeBSD 9 and older are no longer supported. API and Feature Removals ------------------------- +======================== * The ``os.stat_float_times()`` function has been removed. It was introduced in Python 2.3 for backward compatibility with Python 2.2, and was deprecated @@ -1129,15 +2136,39 @@ API and Feature Removals can use attribute access to access items of these dictionaries. * The ``asyncio.windows_utils.socketpair()`` function has been - removed: use directly :func:`socket.socketpair` which is available on all - platforms since Python 3.5 (before, it wasn't available on Windows). + removed. Use the :func:`socket.socketpair` function instead, + it is available on all platforms since Python 3.5. ``asyncio.windows_utils.socketpair`` was just an alias to ``socket.socketpair`` on Python 3.5 and newer. -* :mod:`asyncio`: The module doesn't export :mod:`selectors` and +* :mod:`asyncio` no longer exports the :mod:`selectors` and :mod:`_overlapped` modules as ``asyncio.selectors`` and ``asyncio._overlapped``. Replace ``from asyncio import selectors`` with - ``import selectors`` for example. + ``import selectors``. + +* Direct instantiation of :class:`ssl.SSLSocket` and :class:`ssl.SSLObject` + objects is now prohibited. The constructors were never documented, tested, + or designed as public constructors. Users were suppose to use + :func:`ssl.wrap_socket` or :class:`ssl.SSLContext`. + (Contributed by Christian Heimes in :issue:`32951`.) + + +Windows-only Changes +==================== + +The python launcher, (py.exe), can accept 32 & 64 bit specifiers **without** +having to specify a minor version as well. So ``py -3-32`` and ``py -3-64`` +become valid as well as ``py -3.7-32``, also the -*m*-64 and -*m.n*-64 forms +are now accepted to force 64 bit python even if 32 bit would have otherwise +been used. If the specified version is not available py.exe will error exit. +(Contributed by Steve Barnes in :issue:`30291`.) + +The launcher can be run as ``py -0`` to produce a list of the installed pythons, +*with default marked with an asterisk*. Running ``py -0p`` will include the paths. +If py is run with a version specifier that cannot be matched it will also print +the *short form* list of available specifiers. +(Contributed by Steve Barnes in :issue:`30362`.) + .. _porting-to-python-37: @@ -1148,7 +2179,7 @@ This section lists previously described changes and other bugfixes that may require changes to your code. -Changes in Python behavior +Changes in Python Behavior -------------------------- * :pep:`479` is enabled for all code in Python 3.7, meaning that @@ -1157,6 +2188,13 @@ Changes in Python behavior exceptions. (Contributed by Yury Selivanov in :issue:`32670`.) +* :keyword:`async` and :keyword:`await` names are now reserved keywords. + Code using these names as identifiers will now raise a :exc:`SyntaxError`. + (Contributed by Jelle Zijlstra in :issue:`30406`.) + +* :meth:`object.__aiter__` methods can no longer be declared as + asynchronous. (Contributed by Yury Selivanov in :issue:`31709`.) + * Due to an oversight, earlier Python versions erroneously accepted the following syntax:: @@ -1183,22 +2221,18 @@ Changes in Python behavior Changes in the Python API ------------------------- -* :meth:`socketserver.ThreadingMixIn.server_close` now waits until all - non-daemon threads complete. Use daemonic threads by setting - :data:`socketserver.ThreadingMixIn.daemon_threads` to ``True`` to not - wait until threads complete. - (Contributed by Victor Stinner in :issue:`31233`.) - -* :meth:`socketserver.ForkingMixIn.server_close` now waits until all - child processes complete. (Contributed by Victor Stinner in :issue:`31151`.) +* :meth:`~socketserver.BaseServer.server_close` in + :class:`socketserver.ThreadingMixIn` and :class:`socketserver.ForkingMixIn` + now waits until all non-daemon threads complete. + (Contributed by Victor Stinner in :issue:`31233` and :issue:`31151`.) -* The :func:`locale.localeconv` function now sets temporarily the ``LC_CTYPE`` - locale to the ``LC_NUMERIC`` locale in some cases. +* The :func:`locale.localeconv` function now temporarily sets the ``LC_CTYPE`` + locale to the value of ``LC_NUMERIC`` in some cases. (Contributed by Victor Stinner in :issue:`31900`.) -* :meth:`pkgutil.walk_packages` now raises :exc:`ValueError` if *path* is a string. - Previously an empty list was returned. (Contributed by Sanyam Khurana in - :issue:`24744`.) +* :meth:`pkgutil.walk_packages` now raises a :exc:`ValueError` if *path* is + a string. Previously an empty list was returned. + (Contributed by Sanyam Khurana in :issue:`24744`.) * A format string argument for :meth:`string.Formatter.format` is now :ref:`positional-only `. @@ -1214,7 +2248,7 @@ Changes in the Python API (Contributed by Serhiy Storchaka in :issue:`29192`.) * ``Module``, ``FunctionDef``, ``AsyncFunctionDef``, and - ``ClassDef`` AST nodes now have a new ``docstring`` field. + ``ClassDef`` AST nodes now have the new ``docstring`` attribute. The first statement in their body is not considered as a docstring anymore. ``co_firstlineno`` and ``co_lnotab`` of code object for class and module are affected by this change. (Contributed by INADA Naoki and @@ -1229,34 +2263,35 @@ Changes in the Python API * The :attr:`struct.Struct.format` type is now :class:`str` instead of :class:`bytes`. (Contributed by Victor Stinner in :issue:`21071`.) -* :func:`~cgi.parse_multipart` returns now the same results as +* :func:`~cgi.parse_multipart` now accepts the *encoding* and *errors* + arguments and returns the same results as :class:`~FieldStorage`: for non-file fields, the value associated to a key is a list of strings, not bytes. (Contributed by Pierre Quentel in :issue:`29979`.) -* Due to internal changes in :mod:`socket` you won't be able to - :func:`socket.fromshare` a socket :func:`~socket.socket.share`-ed in older - Python versions. +* Due to internal changes in :mod:`socket` calling :func:`socket.fromshare` + on a socket created by :func:`socket.share ` in older + Python versions is not supported. -* ``repr`` for :exc:`BaseException` has changed not to include trailing comma - in the output. Mind that most exceptions are affected by this change. +* ``repr`` for :exc:`BaseException` has changed not to include the trailing + comma. Most exceptions are affected by this change. (Contributed by Serhiy Storchaka in :issue:`30399`.) -* ``repr`` for :class:`datetime.timedelta` has changed to include keyword arguments - in the output. (Contributed by Utkarsh Upadhyay in :issue:`30302`.) +* ``repr`` for :class:`datetime.timedelta` has changed to include the keyword + arguments in the output. (Contributed by Utkarsh Upadhyay in :issue:`30302`.) * Because :func:`shutil.rmtree` is now implemented using the :func:`os.scandir` function, the user specified handler *onerror* is now called with the first argument ``os.scandir`` instead of ``os.listdir`` when listing the direcory is failed. -* Support of nested sets and set operations in regular expressions as in +* Support for nested sets and set operations in regular expressions as in `Unicode Technical Standard #18`_ might be added in the future. This would - change the syntax, so to facilitate this change a :exc:`FutureWarning` will - be raised in ambiguous cases for the time being. + change the syntax. To facilitate this future change a :exc:`FutureWarning` + will be raised in ambiguous cases for the time being. That include sets starting with a literal ``'['`` or containing literal character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To - avoid a warning escape them with a backslash. + avoid a warning, escape them with a backslash. (Contributed by Serhiy Storchaka in :issue:`30349`.) .. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/ @@ -1289,6 +2324,10 @@ Changes in the Python API of escaping all characters other than ASCII letters, numbers, and ``'_'``. (Contributed by Serhiy Storchaka in :issue:`29995`.) +* :func:`os.listdir` and :func:`os.scandir` now emit file name as bytes when + called with a bytes-like argument. + (Contributed by Serhiy Storchaka in :issue:`30879`.) + * :class:`tracemalloc.Traceback` frames are now sorted from oldest to most recent to be more consistent with :mod:`traceback`. (Contributed by Jesse Bakker in :issue:`32121`.) @@ -1308,64 +2347,94 @@ Changes in the Python API previous behaviour, or use :attr:`STARTUPINFO.lpAttributeList `. +* :meth:`importlib.machinery.PathFinder.invalidate_caches` -- which implicitly + affects :func:`importlib.invalidate_caches` -- now deletes entries + in :data:`sys.path_importer_cache` which are set to ``None``. + (Contributed by Brett Cannon in :issue:`33169`.) + +* In :mod:`asyncio`, + :meth:`loop.sock_recv() `, + :meth:`loop.sock_sendall() `, + :meth:`loop.sock_accept() `, + :meth:`loop.getaddrinfo() `, + :meth:`loop.getnameinfo() ` + have been changed to be proper coroutine methods to match their + documentation. Previously, these methods returned :class:`asyncio.Future` + instances. + (Contributed by Yury Selivanov in :issue:`32327`.) + +* :attr:`asyncio.Server.sockets` now returns a copy of the internal list + of server sockets, instead of returning it directly. + (Contributed by Yury Selivanov in :issue:`32662`.) + +* :attr:`Struct.format ` is now a :class:`str` instance + instead of a :class:`bytes` instance. + (Contributed by Victor Stinner in :issue:`21071`.) + Changes in the C API -------------------- -* The function :c:func:`PySlice_GetIndicesEx` is considered not safe for - resizable sequences. If the slice indices are not instances of :class:`int`, - but objects that implement the :meth:`!__index__` method, the sequence can be - resized after passing its length to :c:func:`!PySlice_GetIndicesEx`. This - can lead to returning indices out of the length of the sequence. For - avoiding possible problems use new functions :c:func:`PySlice_Unpack` and - :c:func:`PySlice_AdjustIndices`. - (Contributed by Serhiy Storchaka in :issue:`27867`.) +The function :c:func:`PySlice_GetIndicesEx` is considered unsafe for +resizable sequences. If the slice indices are not instances of :class:`int`, +but objects that implement the :meth:`!__index__` method, the sequence can be +resized after passing its length to :c:func:`!PySlice_GetIndicesEx`. This +can lead to returning indices out of the length of the sequence. For +avoiding possible problems use new functions :c:func:`PySlice_Unpack` and +:c:func:`PySlice_AdjustIndices`. +(Contributed by Serhiy Storchaka in :issue:`27867`.) CPython bytecode changes ------------------------ -* Added two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD`. - (Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) +There are two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD`. +(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) + +The :opcode:`STORE_ANNOTATION` opcode has been removed. +(Contributed by Mark Shannon in :issue:`32550`.) -* Removed the :opcode:`STORE_ANNOTATION` opcode. - (Contributed by Mark Shannon in :issue:`32550`.) + +Windows-only Changes +-------------------- + +The file used to override :data:`sys.path` is now called +``._pth`` instead of ``'sys.path'``. +See :ref:`finding_modules` for more information. +(Contributed by Steve Dower in :issue:`28137`.) Other CPython implementation changes ------------------------------------ -* In preparation for potential future changes to the public CPython runtime - initialization API (see :pep:`432` for an initial, but somewhat outdated, - draft), CPython's internal startup - and configuration management logic has been significantly refactored. While - these updates are intended to be entirely transparent to both embedding - applications and users of the regular CPython CLI, they're being mentioned - here as the refactoring changes the internal order of various operations - during interpreter startup, and hence may uncover previously latent defects, - either in embedding applications, or in CPython itself. - (Initially contributed by Nick Coghlan and Eric Snow as part of - :issue:`22257`, and further updated by Nick, Eric, and Victor Stinner in a - number of other issues). Some known details affected: - - * :c:func:`PySys_AddWarnOptionUnicode` is not currently usable by embedding - applications due to the requirement to create a Unicode object prior to - calling `Py_Initialize`. Use :c:func:`PySys_AddWarnOption` instead. - * warnings filters added by an embedding application with - :c:func:`PySys_AddWarnOption` should now more consistently take precedence - over the default filters set by the interpreter - -* Due to changes in the way the default warnings filters are configured, - setting :c:data:`Py_BytesWarningFlag` to a value greater than one is no longer - sufficient to both emit :exc:`BytesWarning` messages and have them converted - to exceptions. Instead, the flag must be set (to cause the warnings to be - emitted in the first place), and an explicit ``error::BytesWarning`` - warnings filter added to convert them to exceptions. - -* CPython' :mod:`ssl` module requires OpenSSL 1.0.2 or 1.1 compatible libssl. - OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer - supported. LibreSSL is temporarily not supported as well. LibreSSL releases - up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs. +In preparation for potential future changes to the public CPython runtime +initialization API (see :pep:`432` for an initial, but somewhat outdated, +draft), CPython's internal startup +and configuration management logic has been significantly refactored. While +these updates are intended to be entirely transparent to both embedding +applications and users of the regular CPython CLI, they're being mentioned +here as the refactoring changes the internal order of various operations +during interpreter startup, and hence may uncover previously latent defects, +either in embedding applications, or in CPython itself. +(Initially contributed by Nick Coghlan and Eric Snow as part of +:issue:`22257`, and further updated by Nick, Eric, and Victor Stinner in a +number of other issues). Some known details affected: + +* :c:func:`PySys_AddWarnOptionUnicode` is not currently usable by embedding + applications due to the requirement to create a Unicode object prior to + calling `Py_Initialize`. Use :c:func:`PySys_AddWarnOption` instead. + +* warnings filters added by an embedding application with + :c:func:`PySys_AddWarnOption` should now more consistently take precedence + over the default filters set by the interpreter + +Due to changes in the way the default warnings filters are configured, +setting :c:data:`Py_BytesWarningFlag` to a value greater than one is no longer +sufficient to both emit :exc:`BytesWarning` messages and have them converted +to exceptions. Instead, the flag must be set (to cause the warnings to be +emitted in the first place), and an explicit ``error::BytesWarning`` +warnings filter added to convert them to exceptions. + Documentation From 4c08eac7797547afc2b144a6dfd72e48b10ecf4d Mon Sep 17 00:00:00 2001 From: Elvis Pranskevichus Date: Sat, 19 May 2018 12:20:20 -0400 Subject: [PATCH 2/2] bpo-32996: More What's New in Python 3.7 edits Remove redundant content, reclassify some entries, fix typos and address other review feedback. --- Doc/whatsnew/3.7.rst | 807 +++++++++++++++++++------------------------ 1 file changed, 347 insertions(+), 460 deletions(-) diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index d9e73049495f88..09a6eac879c4cc 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -48,12 +48,6 @@ This article explains the new features in Python 3.7, compared to 3.6. For full details, see the :ref:`changelog `. -.. note:: - - Prerelease users should be aware that this document is currently in draft - form. It will be updated substantially as Python 3.7 moves towards release, - so it's worth checking back even after reading earlier versions. - Summary -- Release Highlights ============================= @@ -114,6 +108,16 @@ C API improvements: * :ref:`PEP 539 `, new C API for thread-local storage +Documentation improvements: + +* :ref:`PEP 545 `, Python documentation translations +* New documentation translations: `Japanese `_, + `French `_, and + `Korean `_. + +This release features notable performance improvements in many areas. +The :ref:`whatsnew37-perf` section lists them in detail. + For a list of changes that may affect compatibility with previous Python releases please refer to the :ref:`porting-to-python-37` section. @@ -244,7 +248,7 @@ PEP 553: Built-in ``breakpoint()`` Python 3.7 includes the new built-in :func:`breakpoint` function as an easy and consistent way to enter the Python debugger. -Built-in ``breakpoint()`` calls :func:`sys.breakpointhook`. By default, this +Built-in ``breakpoint()`` calls :func:`sys.breakpointhook`. By default, the latter imports :mod:`pdb` and then calls ``pdb.set_trace()``, but by binding ``sys.breakpointhook()`` to the function of your choosing, ``breakpoint()`` can enter any debugger. Additionally, the environment variable @@ -296,10 +300,10 @@ PEP 562: Customization of Access to Module Attributes ----------------------------------------------------- Python 3.7 allows defining :meth:`__getattr__` on modules and will call -it whenever a module attribute is accessed. +it whenever a module attribute is otherwise not found. -A typical example of where this may be useful is attribute access deprecation -warnings and lazy loading. +A typical example of where this may be useful is module attribute deprecation +and lazy loading. .. seealso:: @@ -323,8 +327,8 @@ to the :mod:`time` module: * :func:`time.time_ns` The new functions are similar in function to the existing functions -without the ``_ns`` suffix, but instead of returning fractional seconds, -they return nanoseconds as integers. +without the ``_ns`` suffix. They differ by returning nanoseconds as +integers instead of fractional seconds. On Linux and Windows the resolution of :func:`time.time_ns` is 3 times better than that of :func:`time.time`. @@ -437,6 +441,27 @@ keeping ``.pyc`` files up-to-date. See :ref:`pyc-invalidation` for more information. +.. _whatsnew37-pep545: + +PEP 545: Python Documentation Translations +------------------------------------------ + +:pep:`545` describes the process of creating and maintaining Python +documentation translations. + +Three new translations have been added: + +- Japanese: https://docs.python.org/ja/ +- French: https://docs.python.org/fr/ +- Korean: https://docs.python.org/ko/ + +.. seealso:: + + :pep:`545` -- Python Documentation Translations + PEP written and implemented by Julien Palard, Inada Naoki, and + Victor Stinner. + + Other Language Changes ====================== @@ -559,220 +584,145 @@ Improved Modules ================ -abc ---- - -Improved conistency of exceptions raised by :func:`issubclass` when called -with a non-class and an abstract base class as the first and second -arguments, respectively. (Contributed by Josh Bronson in :issue:`33018`.) - - -aifc ----- - -Improved exceptions raised for invalid number of channels and sample -width when read an audio file in modules :mod:`aifc`, :mod:`wave` and -:mod:`sunau`. (Contributed by Serhiy Storchaka in :issue:`32056`.) - - argparse -------- -The new :meth:`ArgumentParser.parse_intermixed_args() \ -` method allows -intermixing options and positional arguments. +The new :meth:`ArgumentParser.parse_intermixed_args() +` +method allows intermixing options and positional arguments. (Contributed by paul.j3 in :issue:`14191`.) -:mod:`argparse` subparsers are now required by default. This matches the -behaviour in Python 2. To add an optional subparser, pass -``required=False`` to -:meth:`ArgumentParser.add_subparsers() `. -(Contributed by Anthony Sottile in :issue:`26510`.) - - -ast ---- - -:meth:`ast.literal_eval()` is now more strict. Addition and subtraction of -arbitrary numbers are no longer allowed. -(Contributed by Serhiy Storchaka in :issue:`31778`.) - .. _whatsnew37_asyncio: asyncio ------- -The new :term:`provisional ` :func:`asyncio.run` function can -be used to run a coroutine from synchronous code by automatically creating and -destroying the event loop. -(Contributed by Yury Selivanov in :issue:`32314`.) - -The new :func:`asyncio.create_task` function has been added as a shortcut -to ``asyncio.get_event_loop().create_task()``. -(Contributed by Andrew Svetlov in :issue:`32311`.) - -The new :meth:`loop.start_tls() ` -method can be used to upgrade an existing connection to TLS. -(Contributed by Yury Selivanov in :issue:`23749`.) - -The new :meth:`loop.sock_recv_into() ` -method allows reading data from a socket directly into a provided buffer making -it possible to reduce data copies. -(Contributed by Antoine Pitrou in :issue:`31819`.) - -The new :func:`asyncio.current_task` function returns the currently running -:class:`~asyncio.Task` instance, and the new :func:`asyncio.all_tasks` -function returns a set of all existing ``Task`` instances in a given loop. -The :meth:`Task.current_task() ` and -:meth:`Task.all_tasks() ` methods have been deprecated. -(Contributed by Andrew Svetlov in :issue:`32250`.) - -The new *provisional* :class:`~asyncio.BufferedProtocol` class allows -implementing streaming protocols with manual control over the receive buffer. -(Contributed by Yury Selivanov in :issue:`32251`.) - -The new :func:`asyncio.get_running_loop` function returns the currently -running loop, or ``None`` if no loop is running. This is in contrast with -:func:`asyncio.get_event_loop`, which will *create* a new event loop if none -is running. -(Contributed by Yury Selivanov in :issue:`32269`.) - -The new :meth:`StreamWriter.wait_closed() ` -coroutine method allows waiting until the stream writer is closed. The new -:meth:`StreamWriter.is_closing() ` method -can be used to determine if the writer is closing. -(Contributed by Andrew Svetlov in :issue:`32391`.) - -The new :meth:`loop.sock_sendfile() ` -coroutine method allows sending files using :mod:`os.sendfile` when possible. -(Contributed by Andrew Svetlov in :issue:`32410`.) - -The new :meth:`Task.get_loop() ` and -:meth:`Future.get_loop() ` methods -return the instance of the loop on which a task or a future were created. -(Contributed by Yury Selivanov in :issue:`32415`.) -:meth:`Server.get_loop() ` allows to do the same for -:class:`asyncio.Server` objects. -(Contributed by Srinivas Reddy Thatiparthy in :issue:`32418`.) - -It is now possible to control how instances of :class:`asyncio.Server` begin -serving. Previously, the server would start serving immediately when created. -The new *start_serving* keyword argument to -:meth:`loop.create_server() ` and -:meth:`loop.create_unix_server() `, -as well as :meth:`Server.start_serving() `, and -:meth:`Server.serve_forever() ` -can be used to decouple server instantiaion and serving. The new -:meth:`Server.is_serving() ` method returns ``True`` -if the server is serving. :class:`~asyncio.Server` objects are now -asynchronous context managers:: - - srv = await loop.create_server(...) - - async with srv: - # some code - - # At this point, srv is closed and no longer accepts new connections. - -(Contributed by Yury Selivanov in :issue:`32662`.) - -Callback objects returned by -:func:`loop.call_later() ` -gained the new :meth:`when() ` method which -returns an absolute scheduled callback timestamp. -(Contributed by Andrew Svetlov in :issue:`32741`.) - -TCP sockets on Linux are now created with ``TCP_NODELAY`` flag set by default. -(Contributed by Yury Selivanov and Victor Stinner in :issue:`27456`.) - -The :meth:`loop.create_datagram_endpoint() \ -` method -gained support for Unix sockets. -(Contributed by Quentin Dawans in :issue:`31245`.) - -The :meth:`loop.create_connection() `, -:meth:`loop.create_server() `, -:meth:`loop.create_unix_server() `, and -:meth:`loop.create_accepted_socket() ` -now accept the *ssl_handshake_timeout* keyword argument. -(Contributed by Neil Aspinall in :issue:`29970`.) - -The new :meth:`Handle.cancelled() ` method returns -``True`` if the callback was cancelled. -(Contributed by Marat Sharafutdinov. in :issue:`31943`.) - -The :meth:`loop.create_unix_connection() ` -method now supports now supports passing a :term:`path-like objects ` -as a socket path. (Contributed by Yury Selivanov in :issue:`32066`.) - -The :func:`asyncio.windows_utils.socketpair`` function has been removed, -:func:`socket.socketpair` should be used instead. -(Contributed by Victor Stinner in :issue:`32154`.) - -The asyncio source has been converted to use the -:keyword:`async`/:keyword:`await` syntax. -(Contributed by Andrew Svetlov in :issue:`32193`.) - -Support for directly ``await``-ing instances of :class:`asyncio.Lock` and -other asyncio synchronization primitives has been deprecated. An -asynchronous context manager has to be used in order to acquire and release -the synchronization resource. See :ref:`async-with-locks` for more -information. -(Contributed by Andrew Svetlov in :issue:`32253`.) - -The :meth:`ReadTransport.resume_reading() ` -and :meth:`ReadTransport.pause_reading() ` -are now idempotent. -The new :meth:`ReadTransport.is_reading() ` -method can be used to determine the reading state of the transport. -(Contributed by Yury Selivanov in :issue:`32356`.) - -Exceptions occurring in cancelled tasks are no longer logged. -(Contributed by Yury Selivanov in :issue:`30508`.) - +The :mod:`asyncio` module has received many new features, usability and +:ref:`performance improvements `. Notable changes +include: + +* The new :term:`provisional ` :func:`asyncio.run` function can + be used to run a coroutine from synchronous code by automatically creating and + destroying the event loop. + (Contributed by Yury Selivanov in :issue:`32314`.) + +* The new :func:`asyncio.create_task` function has been added as a shortcut + to ``asyncio.get_event_loop().create_task()``. + (Contributed by Andrew Svetlov in :issue:`32311`.) + +* The new :meth:`loop.start_tls() ` + method can be used to upgrade an existing connection to TLS. + (Contributed by Yury Selivanov in :issue:`23749`.) + +* The new :meth:`loop.sock_recv_into() ` + method allows reading data from a socket directly into a provided buffer making + it possible to reduce data copies. + (Contributed by Antoine Pitrou in :issue:`31819`.) + +* The new :func:`asyncio.current_task` function returns the currently running + :class:`~asyncio.Task` instance, and the new :func:`asyncio.all_tasks` + function returns a set of all existing ``Task`` instances in a given loop. + The :meth:`Task.current_task() ` and + :meth:`Task.all_tasks() ` methods have been deprecated. + (Contributed by Andrew Svetlov in :issue:`32250`.) + +* The new *provisional* :class:`~asyncio.BufferedProtocol` class allows + implementing streaming protocols with manual control over the receive buffer. + (Contributed by Yury Selivanov in :issue:`32251`.) + +* The new :func:`asyncio.get_running_loop` function returns the currently + running loop, and raises a :exc:`RuntimeError` if no loop is running. + This is in contrast with :func:`asyncio.get_event_loop`, which will *create* + a new event loop if none is running. + (Contributed by Yury Selivanov in :issue:`32269`.) + +* The new :meth:`StreamWriter.wait_closed() ` + coroutine method allows waiting until the stream writer is closed. The new + :meth:`StreamWriter.is_closing() ` method + can be used to determine if the writer is closing. + (Contributed by Andrew Svetlov in :issue:`32391`.) + +* The new :meth:`loop.sock_sendfile() ` + coroutine method allows sending files using :mod:`os.sendfile` when possible. + (Contributed by Andrew Svetlov in :issue:`32410`.) + +* The new :meth:`Task.get_loop() ` and + :meth:`Future.get_loop() ` methods + return the instance of the loop on which a task or a future were created. + :meth:`Server.get_loop() ` allows doing the same for + :class:`asyncio.Server` objects. + (Contributed by Yury Selivanov in :issue:`32415` and + Srinivas Reddy Thatiparthy in :issue:`32418`.) + +* It is now possible to control how instances of :class:`asyncio.Server` begin + serving. Previously, the server would start serving immediately when created. + The new *start_serving* keyword argument to + :meth:`loop.create_server() ` and + :meth:`loop.create_unix_server() `, + as well as :meth:`Server.start_serving() `, and + :meth:`Server.serve_forever() ` + can be used to decouple server instantiation and serving. The new + :meth:`Server.is_serving() ` method returns ``True`` + if the server is serving. :class:`~asyncio.Server` objects are now + asynchronous context managers:: + + srv = await loop.create_server(...) + + async with srv: + # some code + + # At this point, srv is closed and no longer accepts new connections. -.. _whatsnew37-asyncio-optimizations: - -Optimizations -~~~~~~~~~~~~~ - -The :func:`asyncio.get_event_loop` function has been reimplemented in C to -make it up to 15 times faster. -(Contributed by Yury Selivanov in :issue:`32296`.) + (Contributed by Yury Selivanov in :issue:`32662`.) -:class:`asyncio.Future` callback management has been optimized. -(Contributed by Yury Selivanov in :issue:`32348`.) +* Callback objects returned by + :func:`loop.call_later() ` + gained the new :meth:`when() ` method which + returns an absolute scheduled callback timestamp. + (Contributed by Andrew Svetlov in :issue:`32741`.) -:func:`asyncio.gather` is now up to 15% faster. -(Contributed by Yury Selivanov in :issue:`32355`.) +* The :meth:`loop.create_datagram_endpoint() \ + ` method + gained support for Unix sockets. + (Contributed by Quentin Dawans in :issue:`31245`.) -:func:`asyncio.sleep` is now up to 2 times faster when the *delay* -argument is zero or negative. -(Contributed by Andrew Svetlov in :issue:`32351`.) +* The :meth:`loop.create_connection() `, + :meth:`loop.create_server() `, + :meth:`loop.create_unix_server() `, and + :meth:`loop.create_accepted_socket() ` + now accept the *ssl_handshake_timeout* keyword argument. + (Contributed by Neil Aspinall in :issue:`29970`.) -The performance overhead of asyncio debug mode has been reduced. -(Contributed by Antoine Pitrou in :issue:`31970`.) +* The new :meth:`Handle.cancelled() ` method returns + ``True`` if the callback was cancelled. + (Contributed by Marat Sharafutdinov in :issue:`31943`.) +* The asyncio source has been converted to use the + :keyword:`async`/:keyword:`await` syntax. + (Contributed by Andrew Svetlov in :issue:`32193`.) -atexit ------- +* The new :meth:`ReadTransport.is_reading() ` + method can be used to determine the reading state of the transport. + Additionally, calls to + :meth:`ReadTransport.resume_reading() ` + and :meth:`ReadTransport.pause_reading() ` + are now idempotent. + (Contributed by Yury Selivanov in :issue:`32356`.) -The :func:`~atexit.atexit` callbacks are now correctly tracked and called for -subinterpreters. -(Contributed by Marcel Plch in :issue:`31901`.) +* Loop methods which accept socket paths now support passing + :term:`path-like objects `. + (Contributed by Yury Selivanov in :issue:`32066`.) -The traceback for :exc:`SystemExit` raised in an ``atexit`` callback -is no longer displayed. -(Contributed by Serhiy Storchaka in :issue:`28994`.) +* In :mod:`asyncio` TCP sockets on Linux are now created with ``TCP_NODELAY`` + flag set by default. + (Contributed by Yury Selivanov and Victor Stinner in :issue:`27456`.) +* Exceptions occurring in cancelled tasks are no longer logged. + (Contributed by Yury Selivanov in :issue:`30508`.) -bdb ---- - -Pdb and other debuggers dependent on :mod:`bdb` will correctly step over -(next command) native coroutines. -(Contributed by Pablo Galindo. in :issue:`32650`.) +Several ``asyncio`` APIs have been +:ref:`deprecated `. binascii @@ -786,44 +736,17 @@ instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) calendar -------- -The class :class:`~calendar.HTMLCalendar` has new class attributes which ease -the customisation of the CSS classes in the produced HTML calendar. +The :class:`~calendar.HTMLCalendar` class has new class attributes which ease +the customization of CSS classes in the produced HTML calendar. (Contributed by Oz Tiram in :issue:`30095`.) -The :func:`~calendar.weekday` function now accepts any value for its *year* -argument (previously only values from ``1`` to ``9999`` were accepted). -(Contributed by Mark Gollahon in :issue:`28281`.) - -:meth:`Calendar.itermonthdates ` -will now consistently raise an exception when a date falls outside of the -``0001-01-01`` through ``9999-12-31`` range. To support applications that -cannot tolerate such exceptions, the new -:meth:`Calendar.itermonthdays3 ` and -:meth:`Calendar.itermonthdays4 ` can be used. -The new methods return tuples and are not restricted by the range supported by -:class:`datetime.date`. -(Contributed by Alexander Belopolsky in :issue:`28292`.) - collections ----------- -The :func:`collections.namedtuple` factory has been reimplemented to rely -on precomputed methods to substantially reduce the use of :func:`exec`. -As a result, the *verbose* parameter and the *_source* attribute of the -named tuple are not longer supported. The benefits include: -1) having a smaller memory footprint for applications using multiple named -tuples, 2) faster creation of the named tuple class (approx 4x to 6x depending -on how it is measured), and 3) minor speed-ups for instance creation. -(Jelle Zijlstra with further improvements by INADA Naoki, Serhiy Storchaka, -and Raymond Hettinger in :issue:`28638`.) - -Additionally, ``collections.namedtuple()`` now supports default values. +``collections.namedtuple()`` now supports default values. (Contributed by Raymond Hettinger in :issue:`32320`.) -:class:`collections.ChainMap` now preserves the order of the underlying -mappings. (Contributed by Raymond Hettinger in :issue:`32792`.) - compileall ---------- @@ -848,120 +771,77 @@ The :class:`ProcessPoolExecutor ` can now take the multiprocessing context via the new *mp_context* argument. (Contributed by Thomas Moreau in :issue:`31540`.) -``executor.submit()`` now raises a :exc:`RuntimeError` if called during -interpreter shutdown. -(Contributed by Mark Nemec in :issue:`33097`.) - - -configparser ------------- - -The :class:`~configparser.ConfigParser`` constructor now uses ``read_dict()`` -to process the default values, making its behavior consistent with the -rest of the parser. Non-string keys and values in the defaults -dictionary are now being implicitly converted to strings. -(Contributed by James Tocknell. in :issue:`23835`.) - -The *filename* parameter of the -:meth:`ConfigParser.read() ` method now -accepts bytes objects. -(Contributed by Vincent Michel. in :issue:`31307`.) - contextlib ---------- -The new :func:`~contextlib.nullcontext` is a simpler and faster alternative -to :class:`~contextlib.ExitStack` for handling simple context managers. +The new :func:`~contextlib.nullcontext` is a simpler and faster no-op +context manager than :class:`~contextlib.ExitStack`. (Contributed by Jesse-Bakker in :issue:`10049`.) -The new :func:`~contextlib.asynccontextmanager` and -:class:`~contextlib.AbstractAsyncContextManager` have been added to +The new :func:`~contextlib.asynccontextmanager`, +:class:`~contextlib.AbstractAsyncContextManager`, and +:class:`~contextlib.AsyncExitStack` have been added to complement their synchronous counterparts. (Contributed -by Jelle Zijlstra in :issue:`29679` and :issue:`30241`.) - -The new :class:`contextlib.AsyncExitStack` class has been added as an -asynchronous equivalent to :class:`contextlib.ExitStack`. (Contributed by -Alexander Mohr and Ilya Kulakov in :issue:`29302`.) +by Jelle Zijlstra in :issue:`29679` and :issue:`30241`, +and by Alexander Mohr and Ilya Kulakov in :issue:`29302`.) cProfile -------- -:mod:`cProfile` command line now accepts ``-m module_name`` as an alternative -to script path. (Contributed by Sanyam Khurana in :issue:`21862`.) +The :mod:`cProfile` command line now accepts ``-m module_name`` as an +alternative to script path. (Contributed by Sanyam Khurana in :issue:`21862`.) crypt ----- -Added support for the Blowfish hashing method. +The :mod:`crypt` module now supports the Blowfish hashing method. (Contributed by Serhiy Storchaka in :issue:`31664`.) -The :func:`~crypt.mksalt` function now allows to specify the number of rounds +The :func:`~crypt.mksalt` function now allows specifying the number of rounds for hashing. (Contributed by Serhiy Storchaka in :issue:`31702`.) -csv ---- - -A single empty field is now always quoted when written into a CSV -file. This allows to distinguish an empty row from a row consisting of -a single empty field. -(Contributed by Licht Takeuchi. in :issue:`32255`.) - - datetime -------- -Added the :meth:`datetime.fromisoformat() ` -method, which constructs a :class:`~datetime.datetime` object from a string +The new :meth:`datetime.fromisoformat() ` +method constructs a :class:`~datetime.datetime` object from a string in one of the formats output by :meth:`datetime.isoformat() `. (Contributed by Paul Ganssle in :issue:`15873`.) -The :class:`tzinfo ` objects now supprt sub-minute offsets. +The :class:`tzinfo ` class now supports sub-minute offsets. (Contributed by Alexander Belopolsky in :issue:`5288`.) dbm --- -:func:`dbm.dumb` now supports reading read-only files and no longer writes the -index file when it is not changed. A deprecation warning is now emitted -if the index file is missing and recreated in the ``'r'`` and ``'w'`` -modes (this will be an error in future Python releases). -(Contributed by Serhiy Storchaka in :issue:`28847`.) - - -:func:`dbm.open` now encodes filenames using the filesystem encoding rather -than default encoding. -(Contributed by Serhiy Storchaka in :issue:`32922`.) +:mod:`dbm.dumb` now supports reading read-only files and no longer writes the +index file when it is not changed. decimal ------- -The :mod:`decimal` now uses :ref:`context variables ` to -store the decimal context. +The :mod:`decimal` module now uses :ref:`context variables ` +to store the decimal context. (Contributed by Yury Selivanov in :issue:`32630`.) dis --- -The :func:`~dis.dis` function now is able to +The :func:`~dis.dis` function is now able to disassemble nested code objects (the code of comprehensions, generator expressions and nested functions, and the code used for building nested classes). The maximum depth of disassembly recursion is controlled by the new *depth* parameter. (Contributed by Serhiy Storchaka in :issue:`11822`.) -Functions in the :mod:`dis` module now works with asynchronous generators and -coroutine objects. -(Contributed by George Collins based on diagnosis by Luciano Ramalho -in :issue:`31183`.) - distutils --------- @@ -970,18 +850,6 @@ distutils therefore included in source distributions. (Contributed by Ryan Gonzalez in :issue:`11913`.) -:class:`distutils.core.setup` now warns if the ``classifiers``, ``keywords`` -and ``platforms`` fields are not specified as a list or a string. -(Contributed by Berker Peksag in :issue:`19610`.) - -The ``upload`` command no longer tries to change CR end-of-line characters -to CRLF. This fixes a corruption issue with sdists that ended with a byte -equivalent to CR. -(Contributed by Bo Bayles in :issue:`32304`.) - -The unused ``install_misc`` command has been removed. -(Contributed by Eric N. Vander Weele. in :issue:`29218`.) - enum ---- @@ -999,15 +867,6 @@ return :const:`False` instead and are deprecated. (Contributed by Ethan Furman in :issue:`33217`.) -fpectl ------- - -The ``fpectl`` module has been removed. It was never enabled by -default, never worked correctly on x86-64, and it changed the Python -ABI in ways that caused unexpected breakage of C extensions. -(Contributed by Nathaniel J. Smith in :issue:`29137`.) - - functools --------- @@ -1019,8 +878,8 @@ using type annotations. gc -- -The new :func:`gc.freeze`` function allows freezing all objects tracked -by the garbage collector and exclude them from future collections. +The new :func:`gc.freeze` function allows freezing all objects tracked +by the garbage collector and excluding them from future collections. This can be used before a POSIX ``fork()`` call to make the GC copy-on-write friendly or to speed up collection. The new :func:`gc.unfreeze` functions reverses this operation. Additionally, :func:`gc.get_freeze_count` can @@ -1064,13 +923,6 @@ It is used when ``http.server`` is run with ``-m``. (Contributed by Julien Palard in :issue:`31639`.) -idlelib and IDLE ----------------- - -IDLE module browser now shows nested classes and functions. -(Contributed by Guilherme Polo and Cheryl Sabella in :issue:`1612262`.) - - importlib --------- @@ -1105,10 +957,10 @@ INADA Naoki in :issue:`15216`.) ipaddress --------- -The new ``subnet_of`` and ``superset_of`` methods of +The new ``subnet_of()`` and ``supernet_of()`` methods of :class:`ipaddress.IPv6Network` and :class:`ipaddress.IPv4Network` can be used for network containment tests. -(Contributed by Michel Albert and Cheryl Sabella. in :issue:`20825`.) +(Contributed by Michel Albert and Cheryl Sabella in :issue:`20825`.) itertools @@ -1127,8 +979,6 @@ The new *monetary* argument to :func:`locale.format_string` can be used to make the conversion use monetary thousands separators and grouping strings. (Contributed by Garvit in :issue:`10379`.) -The :func:`locale.format` function is deprecated. - The :func:`locale.getpreferredencoding` function now always returns ``'UTF-8'`` on Android or when in the :ref:`forced UTF-8 mode `. @@ -1139,23 +989,15 @@ logging :class:`~logging.Logger` instances can now be pickled. (Contributed by Vinay Sajip in :issue:`30520`.) -The new :meth:`StreamHandler.setStream() `` +The new :meth:`StreamHandler.setStream() ` method can be used to replace the logger stream after handler creation. (Contributed by Vinay Sajip in :issue:`30522`.) -It is now possible to specify keyword arguments to handler constructor in +It is now possible to specify keyword arguments to handler constructors in configuration passed to :func:`logging.config.fileConfig`. (Contributed by Preston Landers in :issue:`31080`.) -macpath -------- - -The :mod:`macpath` is now deprecated and will be removed in Python -3.8. -(Contributed by Chi Hsuan Yen in :issue:`9850`.) - - math ---- @@ -1207,7 +1049,7 @@ os (Contributed by Serhiy Storchaka in :issue:`25996`.) The new :func:`~os.register_at_fork` function allows registering Python -callbacks to be executed on a process fork. +callbacks to be executed at process fork. (Contributed by Antoine Pitrou in :issue:`16500`.) Exposed the *preadv*, *preadv2*, *pwritev* and *pwritev2* system calls through @@ -1235,10 +1077,6 @@ on POSIX systems and can be used to determine whether a path is a mount point. (Contributed by Cooper Ry Lees in :issue:`30897`.) -Exposed the system call *posix_spawn* through the new function -:func:`~os.posix_spawn`. (Contributed by Pablo Galindo, Serhiy Storchaka and -Gregory P. Smith in :issue:`20104`.) - pdb --- @@ -1319,7 +1157,7 @@ The new :func:`socket.getblocking() ` method returns ``True`` if the socket is in blocking mode and ``False`` otherwise. (Contributed by Yury Selivanov in :issue:`32373`.) -The new :func:`socket.close` function closes a passed socket file descriptor. +The new :func:`socket.close` function closes the passed socket file descriptor. This function should be used instead of :func:`os.close` for better compatibility across platforms. (Contributed by Christian Heimes in :issue:`32454`.) @@ -1327,8 +1165,8 @@ compatibility across platforms. The :mod:`socket` module now exposes the :data:`socket.TCP_CONGESTION` (Linux 2.6.13), :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and :data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. -(Contributed by Omar Sandoval. in :issue:`26273` and -Nathaniel J. Smith. in :issue:`29728`.) +(Contributed by Omar Sandoval in :issue:`26273` and +Nathaniel J. Smith in :issue:`29728`.) Support for :data:`socket.AF_VSOCK` sockets has been added to allow communication between virtual machines and their hosts. @@ -1365,9 +1203,10 @@ Host name validation can be customized with (Contributed by Christian Heimes in :issue:`31399`.) .. note:: - The improved host name check requires OpenSSL 1.0.2 or 1.1 compatible - libssl. OpenSSL 0.9.8 and 1.0.1 are no longer supported. LibreSSL is - temporarily not supported until it gains the necessary OpenSSL 1.0.2 APIs. + The improved host name check requires a *libssl* implementation compatible + with OpenSSL 1.0.2 or 1.1. Consequently, OpenSSL 0.9.8 and 1.0.1 are no + longer supported and LibreSSL is temporarily not supported until it gains + the necessary OpenSSL 1.0.2 APIs. The ``ssl`` module no longer sends IP addresses in SNI TLS extension. (Contributed by Christian Heimes in :issue:`32185`.) @@ -1421,7 +1260,7 @@ subprocess ---------- The :func:`subprocess.run` function accepts the new *capture_output* -keyword argument. When ``True``, stdout and stderr will be captured. +keyword argument. When true, stdout and stderr will be captured. This is equivalent to passing :data:`subprocess.PIPE` as *stdout* and *stderr* arguments. (Contributed by Bo Bayles in :issue:`32102`.) @@ -1433,7 +1272,7 @@ to *universal_newlines*. On Windows the default for *close_fds* was changed from ``False`` to ``True`` when redirecting the standard handles. It's now possible to set -*close_fds* to ``True`` when redirecting the standard handles. See +*close_fds* to true when redirecting the standard handles. See :class:`subprocess.Popen`. This means that *close_fds* now defaults to ``True`` on all supported platforms. (Contributed by Segev Finer in :issue:`19764`.) @@ -1450,8 +1289,6 @@ exception. sys --- -Added :attr:`sys.flags.dev_mode` flag for the new development mode. - The new :func:`sys.breakpointhook` hook function is called by the built-in :func:`breakpoint`. (Contributed by Barry Warsaw in :issue:`31353`.) @@ -1462,7 +1299,9 @@ Android API version. The new :func:`sys.get_coroutine_origin_tracking_depth` function returns the current coroutine origin tracking depth, as set by -the new :func:`sys.set_coroutine_origin_tracking_depth`. +the new :func:`sys.set_coroutine_origin_tracking_depth`. :mod:`asyncio` +has been converted to use this new API instead of +the deprecated :func:`sys.set_coroutine_wrapper`. (Contributed by Nathaniel J. Smith in :issue:`32591`.) @@ -1485,10 +1324,10 @@ New clock identifiers have been added: :data:`time.CLOCK_MONOTONIC`, except it also includes any time that the system is suspended. * :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution - per-process timer from the CPU. + per-process CPU timer. * :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is the time the system has been running and not suspended, providing accurate - uptime measurement, both absolute and interval. + uptime measurement. The new :func:`time.thread_time` and :func:`time.thread_time_ns` functions can be used to get per-thread CPU time measurements. @@ -1498,16 +1337,6 @@ The new :func:`time.pthread_getcpuclockid` function returns the clock ID of the thread-specific CPU-time clock. -timeit ------- - -:func:`timeit.timeit` autorange now uses a single loop iteration if -the benchmark takes less than 10 seconds, instead of 10 iterations. -``python3 -m timeit -s 'import time' 'time.sleep(1)'`` now takes 4 seconds -instead of 40 seconds. -(Contributed by Victor Stinner in :issue:`28240`.) - - tkinter ------- @@ -1524,20 +1353,17 @@ sorting the frames from oldest to most recent. now accepts negative *limit*, truncating the result to the ``abs(limit)`` oldest frames. To get the old behaviour, use the new *most_recent_first* argument to ``Traceback.format()``. -(Contributed by Jesse Bakker. in :issue:`32121`.) +(Contributed by Jesse Bakker in :issue:`32121`.) types ----- The new :class:`~types.WrapperDescriptorType`, -:class:`~types.MethodWrapperType`, and :class:`~types.MethodDescriptorType` -classes are now available. -(Contributed by Manuel Krebber and Guido van Rossum in :issue:`29377`.) - -The new :class:`~types.ClassMethodDescriptorType` is used to describe -class and static methods of builtin types. -(Contributed by Serhiy Storchaka in :issue:`32265`.) +:class:`~types.MethodWrapperType`, :class:`~types.MethodDescriptorType`, +and :class:`~types.ClassMethodDescriptorType` classes are now available. +(Contributed by Manuel Krebber and Guido van Rossum in :issue:`29377`, +and Serhiy Storchaka in :issue:`32265`.) The new :func:`types.resolve_bases` function resolves MRO entries dynamically as specified by :pep:`560`. @@ -1555,9 +1381,10 @@ Peterson.) unittest -------- -Added new command-line option ``-k`` to filter tests to run with a substring or -Unix shell-like pattern. For example, ``python -m unittest -k foo`` runs the -tests ``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``, +The new ``-k`` command-line option allows filtering tests by a name +substring or a Unix shell-like pattern. +For example, ``python -m unittest -k foo`` runs +``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``, but not ``bar_tests.FooTest.test_something``. (Contributed by Jonas Haag in :issue:`32071`.) @@ -1569,24 +1396,25 @@ The :const:`~unittest.mock.sentinel` attributes now preserve their identity when they are :mod:`copied ` or :mod:`pickled `. (Contributed by Serhiy Storchaka in :issue:`20804`.) -New function :func:`~unittest.mock.seal` will disable the creation of mock -children by preventing to get or set any new attribute on the sealed mock. -The sealing process is performed recursively. (Contributed by Mario Corchero -in :issue:`30541`.) +The new :func:`~unittest.mock.seal` function allows sealing +:class:`~unittest.mock.Mock` instances, which will disallow further creation +of attribute mocks. The seal is applied recursively to all attributes that +are themselves mocks. +(Contributed by Mario Corchero in :issue:`30541`.) urllib.parse ------------ :func:`urllib.parse.quote` has been updated from :rfc:`2396` to :rfc:`3986`, -adding ``~`` to the set of characters that is never quoted by default. +adding ``~`` to the set of characters that are never quoted by default. (Contributed by Christian Theune and Ratnadeep Debnath in :issue:`16285`.) uu -- -Function :func:`~uu.encode` now accepts an optional *backtick* +The :func:`uu.encode` function now accepts an optional *backtick* keyword argument. When it's true, zeros are represented by ``'`'`` instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) @@ -1613,27 +1441,28 @@ warnings The initialization of the default warnings filters has changed as follows: * warnings enabled via command line options (including those for :option:`-b` - and the new CPython-specific ``-X dev`` option) are always passed to the - warnings machinery via the ``sys.warnoptions`` attribute. + and the new CPython-specific :option:`-X` ``dev`` option) are always passed + to the warnings machinery via the :data:`sys.warnoptions` attribute. + * warnings filters enabled via the command line or the environment now have the - following precedence order: + following order of precedence: * the ``BytesWarning`` filter for :option:`-b` (or ``-bb``) - * any filters specified with :option:`-W` - * any filters specified with :envvar:`PYTHONWARNINGS` + * any filters specified with the :option:`-W` option + * any filters specified with the :envvar:`PYTHONWARNINGS` environment + variable * any other CPython specific filters (e.g. the ``default`` filter added for the new ``-X dev`` mode) * any implicit filters defined directly by the warnings machinery + * in CPython debug builds, all warnings are now displayed by default (the implicit filter list is empty) (Contributed by Nick Coghlan and Victor Stinner in :issue:`20361`, -:issue:`32043`, and :issue:`32230`) +:issue:`32043`, and :issue:`32230`.) -The default warning filter list now starts with a -``"default::DeprecationWarning:__main__"`` entry, so deprecation warnings -are once again shown by default in single-file scripts and at the -interactive prompt. See :ref:`whatsnew37-pep565` for details. +Deprecation warnings are once again shown by default in single-file scripts and +at the interactive prompt. See :ref:`whatsnew37-pep565` for details. (Contributed by Nick Coghlan in :issue:`31975`.) @@ -1672,7 +1501,7 @@ zipfile :class:`~zipfile.ZipFile` now accepts the new *compresslevel* parameter to control the compression level. -(Contributed by Stephen J. Turnbull. in :issue:`28102`.) +(Contributed by Bo Bayles in :issue:`21417`.) Subdirectories in archives created by ``ZipFile`` are now stored in alphabetical order. @@ -1772,7 +1601,7 @@ now use the current locale encoding for ``surrogateescape`` error handler. (Contributed by Victor Stinner in :issue:`29240`.) The *start* and *end* parameters of :c:func:`PyUnicode_FindChar` are -now adjusted to behave like ``str[start:end]``. +now adjusted to behave like string slices. (Contributed by Xiang Zhang in :issue:`28822`.) @@ -1800,6 +1629,8 @@ supported. LibreSSL is temporarily not supported as well. LibreSSL releases up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs. +.. _whatsnew37-perf: + Optimizations ============= @@ -1809,18 +1640,41 @@ to use the ``METH_FASTCALL`` convention. (Contributed by Victor Stinner in :issue:`29300`, :issue:`29507`, :issue:`29452`, and :issue:`29286`.) -Various optimizations have reduced Python startup time by 5% on Linux and +Various optimizations have reduced Python startup time by 10% on Linux and up to 30% on macOS. -(Contributed by Victor Stinner, INADA Naoki in :issue:`29585`.) +(Contributed by Victor Stinner, INADA Naoki in :issue:`29585`, and +Ivan Levkivskyi in :issue:`31333`.) Method calls are now up to 20% faster due to the bytecode changes which avoid creating bound method instances. (Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) -The :mod:`asyncio` received a number of notable optimizations for commonly -used functions. -See :ref:`optimizations in asyncio ` for -details. +.. _whatsnew37-asyncio-perf: + +The :mod:`asyncio` module received a number of notable optimizations for +commonly used functions: + +* The :func:`asyncio.get_event_loop` function has been reimplemented in C to + make it up to 15 times faster. + (Contributed by Yury Selivanov in :issue:`32296`.) + +* :class:`asyncio.Future` callback management has been optimized. + (Contributed by Yury Selivanov in :issue:`32348`.) + +* :func:`asyncio.gather` is now up to 15% faster. + (Contributed by Yury Selivanov in :issue:`32355`.) + +* :func:`asyncio.sleep` is now up to 2 times faster when the *delay* + argument is zero or negative. + (Contributed by Andrew Svetlov in :issue:`32351`.) + +* The performance overhead of asyncio debug mode has been reduced. + (Contributed by Antoine Pitrou in :issue:`31970`.) + +As a result of :ref:`PEP 560 work `, the import time +of :mod:`typing` has been reduced by a factor of 7, and many typing operations +are now faster. +(Contributed by Ivan Levkivskyi in :issue:`32226`.) :func:`sorted` and :meth:`list.sort` have been optimized for common cases to be up to 40-75% faster. @@ -1829,7 +1683,7 @@ to be up to 40-75% faster. :meth:`dict.copy` is now up to 5.5 times faster. (Contributed by Yury Selivanov in :issue:`31179`.) -:func:`hasattr` and :func:`getattr`` are now about 4 times faster when +:func:`hasattr` and :func:`getattr` are now about 4 times faster when *name* is not found and *obj* does not override :meth:`object.__getattr__` or :meth:`object.__getattribute__`. (Contributed by INADA Naoki in :issue:`32544`.) @@ -1848,10 +1702,6 @@ Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) 30% faster in the common case. (Contributed by Paul Ganssle in :issue:`32403`.) -Fast implementation from standard C library is now used for the -:func:`math.erf` and :func:`math.erfc` functions. -(Contributed by Serhiy Storchaka in :issue:`26121`.) - The :func:`os.fwalk` function is now up to 2 times faster thanks to the use of :func:`os.scandir`. (Contributed by Serhiy Storchaka in :issue:`25996`.) @@ -1903,10 +1753,10 @@ C library implementation on most platforms. Other CPython Implementation Changes ==================================== -* Trace hooks may now opt out of receiving the ``line`` and ``opcode`` events - from the interpreter by setting the corresponding new ``f_trace_lines`` and - ``f_trace_opcodes`` attributes to :const:`False` on the frame - being traced. (Contributed by Nick Coghlan in :issue:`31344`.) +* Trace hooks may now opt out of receiving the ``line`` and opt into receiving + the ``opcode`` events from the interpreter by setting the corresponding new + ``f_trace_lines`` and ``f_trace_opcodes`` attributes on the + frame being traced. (Contributed by Nick Coghlan in :issue:`31344`.) * Fixed some consistency problems with namespace package module attributes. Namespace module objects now have an ``__file__`` that is set to ``None`` @@ -1917,9 +1767,14 @@ Other CPython Implementation Changes :issue:`32303`. * The :func:`locals` dictionary now displays in the lexical order that - variables were defined. Previously, the order was reversed. + variables were defined. Previously, the order was undefined. (Contributed by Raymond Hettinger in :issue:`32690`.) +* The :mod:`distutils` ``upload`` command no longer tries to change CR + end-of-line characters to CRLF. This fixes a corruption issue with sdists + that ended with a byte equivalent to CR. + (Contributed by Bo Bayles in :issue:`32304`.) + Deprecated Python Behavior ========================== @@ -1954,12 +1809,14 @@ Use :func:`aifc.open` instead. (Contributed by Brian Curtin in :issue:`31985`.) +.. _whatsnew37-asyncio-deprecated: + asyncio ------- Support for directly ``await``-ing instances of :class:`asyncio.Lock` and other asyncio synchronization primitives has been deprecated. An -asynchronous context manager has to be used in order to acquire and release +asynchronous context manager must be used in order to acquire and release the synchronization resource. See :ref:`async-with-locks` for more information. (Contributed by Andrew Svetlov in :issue:`32253`.) @@ -1979,6 +1836,16 @@ base classes. (Contributed by Serhiy Storchaka in :issue:`25988`.) +dbm +--- + +:mod:`dbm.dumb` now supports reading read-only files and no longer writes the +index file when it is not changed. A deprecation warning is now emitted +if the index file is missing and recreated in the ``'r'`` and ``'w'`` +modes (this will be an error in future Python releases). +(Contributed by Serhiy Storchaka in :issue:`28847`.) + + enum ---- @@ -2027,6 +1894,7 @@ macpath ------- The :mod:`macpath` is now deprecated and will be removed in Python 3.8. +(Contributed by Chi Hsuan Yen in :issue:`9850`.) threading @@ -2086,9 +1954,9 @@ Deprecated functions and types of the C API =========================================== Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with -a macro if ``Py_LIMITED_API`` is not set or set to the value between -``0x03050400`` and ``0x03060000`` (not including) or ``0x03060100`` or -higher. (Contributed by Serhiy Storchaka in :issue:`27867`.) +a macro if ``Py_LIMITED_API`` is not set or set to a value in the range +between ``0x03050400`` and ``0x03060000`` (not inclusive), or is ``0x03060100`` +or higher. (Contributed by Serhiy Storchaka in :issue:`27867`.) :c:func:`PyOS_AfterFork` has been deprecated. Use :c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` or :c:func:`PyOS_AfterFork_Child()` instead. @@ -2104,6 +1972,8 @@ FreeBSD 9 and older are no longer officially supported. API and Feature Removals ======================== +The following features and APIs have been removed from Python 3.7: + * The ``os.stat_float_times()`` function has been removed. It was introduced in Python 2.3 for backward compatibility with Python 2.2, and was deprecated since Python 3.1. @@ -2148,10 +2018,22 @@ API and Feature Removals * Direct instantiation of :class:`ssl.SSLSocket` and :class:`ssl.SSLObject` objects is now prohibited. The constructors were never documented, tested, - or designed as public constructors. Users were suppose to use + or designed as public constructors. Users were supposed to use :func:`ssl.wrap_socket` or :class:`ssl.SSLContext`. (Contributed by Christian Heimes in :issue:`32951`.) +* The unused :mod:`distutils` ``install_misc`` command has been removed. + (Contributed by Eric N. Vander Weele in :issue:`29218`.) + + +Module Removals +=============== + +The ``fpectl`` module has been removed. It was never enabled by +default, never worked correctly on x86-64, and it changed the Python +ABI in ways that caused unexpected breakage of C extensions. +(Contributed by Nathaniel J. Smith in :issue:`29137`.) + Windows-only Changes ==================== @@ -2182,16 +2064,16 @@ that may require changes to your code. Changes in Python Behavior -------------------------- +* :keyword:`async` and :keyword:`await` names are now reserved keywords. + Code using these names as identifiers will now raise a :exc:`SyntaxError`. + (Contributed by Jelle Zijlstra in :issue:`30406`.) + * :pep:`479` is enabled for all code in Python 3.7, meaning that :exc:`StopIteration` exceptions raised directly or indirectly in coroutines and generators are transformed into :exc:`RuntimeError` exceptions. (Contributed by Yury Selivanov in :issue:`32670`.) -* :keyword:`async` and :keyword:`await` names are now reserved keywords. - Code using these names as identifiers will now raise a :exc:`SyntaxError`. - (Contributed by Jelle Zijlstra in :issue:`30406`.) - * :meth:`object.__aiter__` methods can no longer be declared as asynchronous. (Contributed by Yury Selivanov in :issue:`31709`.) @@ -2221,6 +2103,13 @@ Changes in Python Behavior Changes in the Python API ------------------------- +* ``Module``, ``FunctionDef``, ``AsyncFunctionDef``, and + ``ClassDef`` AST nodes now have the new ``docstring`` attribute. + The first statement in their body is not considered as a docstring + anymore. ``co_firstlineno`` and ``co_lnotab`` of code object for class + and module are affected by this change. (Contributed by INADA Naoki and + Eugene Toder in :issue:`29463`.) + * :meth:`~socketserver.BaseServer.server_close` in :class:`socketserver.ThreadingMixIn` and :class:`socketserver.ForkingMixIn` now waits until all non-daemon threads complete. @@ -2247,13 +2136,6 @@ Changes in the Python API Use the :meth:`~http.cookies.Morsel.set` method for setting them. (Contributed by Serhiy Storchaka in :issue:`29192`.) -* ``Module``, ``FunctionDef``, ``AsyncFunctionDef``, and - ``ClassDef`` AST nodes now have the new ``docstring`` attribute. - The first statement in their body is not considered as a docstring - anymore. ``co_firstlineno`` and ``co_lnotab`` of code object for class - and module are affected by this change. (Contributed by INADA Naoki and - Eugene Toder in :issue:`29463`.) - * The *mode* argument of :func:`os.makedirs` no longer affects the file permission bits of newly-created intermediate-level directories. To set their file permission bits you can set the umask before invoking @@ -2269,11 +2151,11 @@ Changes in the Python API is a list of strings, not bytes. (Contributed by Pierre Quentel in :issue:`29979`.) -* Due to internal changes in :mod:`socket` calling :func:`socket.fromshare` +* Due to internal changes in :mod:`socket`, calling :func:`socket.fromshare` on a socket created by :func:`socket.share ` in older Python versions is not supported. -* ``repr`` for :exc:`BaseException` has changed not to include the trailing +* ``repr`` for :exc:`BaseException` has changed to not include the trailing comma. Most exceptions are affected by this change. (Contributed by Serhiy Storchaka in :issue:`30399`.) @@ -2324,10 +2206,6 @@ Changes in the Python API of escaping all characters other than ASCII letters, numbers, and ``'_'``. (Contributed by Serhiy Storchaka in :issue:`29995`.) -* :func:`os.listdir` and :func:`os.scandir` now emit file name as bytes when - called with a bytes-like argument. - (Contributed by Serhiy Storchaka in :issue:`30879`.) - * :class:`tracemalloc.Traceback` frames are now sorted from oldest to most recent to be more consistent with :mod:`traceback`. (Contributed by Jesse Bakker in :issue:`32121`.) @@ -2371,6 +2249,40 @@ Changes in the Python API instead of a :class:`bytes` instance. (Contributed by Victor Stinner in :issue:`21071`.) +* :mod:`argparse` subparsers are now required by default. This matches the + behaviour in Python 2. To add an optional subparser, pass + ``required=False`` to + :meth:`ArgumentParser.add_subparsers() `. + (Contributed by Anthony Sottile in :issue:`26510`.) + +* :meth:`ast.literal_eval()` is now stricter. Addition and subtraction of + arbitrary numbers are no longer allowed. + (Contributed by Serhiy Storchaka in :issue:`31778`.) + +* :meth:`Calendar.itermonthdates ` + will now consistently raise an exception when a date falls outside of the + ``0001-01-01`` through ``9999-12-31`` range. To support applications that + cannot tolerate such exceptions, the new + :meth:`Calendar.itermonthdays3 ` and + :meth:`Calendar.itermonthdays4 ` can be used. + The new methods return tuples and are not restricted by the range supported by + :class:`datetime.date`. + (Contributed by Alexander Belopolsky in :issue:`28292`.) + +* :class:`collections.ChainMap` now preserves the order of the underlying + mappings. (Contributed by Raymond Hettinger in :issue:`32792`.) + +* The ``submit()`` method of :class:`concurrent.futures.ThreadPoolExecutor` + and :class:`concurrent.futures.ProcessPoolExecutor` now raises + a :exc:`RuntimeError` if called during interpreter shutdown. + (Contributed by Mark Nemec in :issue:`33097`.) + +* The :class:`configparser.ConfigParser` constructor now uses ``read_dict()`` + to process the default values, making its behavior consistent with the + rest of the parser. Non-string keys and values in the defaults + dictionary are now being implicitly converted to strings. + (Contributed by James Tocknell in :issue:`23835`.) + Changes in the C API -------------------- @@ -2434,28 +2346,3 @@ sufficient to both emit :exc:`BytesWarning` messages and have them converted to exceptions. Instead, the flag must be set (to cause the warnings to be emitted in the first place), and an explicit ``error::BytesWarning`` warnings filter added to convert them to exceptions. - - - -Documentation -============= - -.. _whatsnew37-pep545: - -PEP 545: Python Documentation Translations ------------------------------------------- - -:pep:`545` describes the process to translate Python documentation, -and three translations have been added: - -- Japanese: https://docs.python.org/ja/ and associated GitHub - repository: https://github.com/python/python-docs-ja - -- French: https://docs.python.org/fr/ and associated GitHub - repository: https://github.com/python/python-docs-fr - -- Korean: https://docs.python.org/ko/ and associated GitHub - repository: https://github.com/python/python-docs-ko - -(Contributed by Julien Palard, Inada Naoki, and Victor Stinner in -:issue:`26546`.)