``).
The *tag* argument is the name of the tag converted to lower case. The *attrs*
argument is a list of ``(name, value)`` pairs containing the attributes found
diff --git a/Doc/library/http.cookiejar.rst b/Doc/library/http.cookiejar.rst
index 9ac5d52a2ab09c..fb113d89e97535 100644
--- a/Doc/library/http.cookiejar.rst
+++ b/Doc/library/http.cookiejar.rst
@@ -122,7 +122,7 @@ The following classes are provided:
:mod:`http.cookiejar` and :mod:`http.cookies` modules do not depend on each
other.
- https://curl.haxx.se/rfc/cookie_spec.html
+ https://curl.se/rfc/cookie_spec.html
The specification of the original Netscape cookie protocol. Though this is
still the dominant protocol, the 'Netscape cookie protocol' implemented by all
the major browsers (and :mod:`http.cookiejar`) only bears a passing resemblance to
diff --git a/Doc/library/http.server.rst b/Doc/library/http.server.rst
index 478a5b31475cfd..94647e99724b33 100644
--- a/Doc/library/http.server.rst
+++ b/Doc/library/http.server.rst
@@ -20,7 +20,7 @@ This module defines classes for implementing HTTP servers (Web servers).
.. warning::
:mod:`http.server` is not recommended for production. It only implements
- basic security checks.
+ :ref:`basic security checks `.
One class, :class:`HTTPServer`, is a :class:`socketserver.TCPServer` subclass.
It creates and listens at the HTTP socket, dispatching the requests to a
@@ -98,7 +98,9 @@ provides three different variants:
.. attribute:: path
- Contains the request path.
+ Contains the request path. If query component of the URL is present,
+ then ``path`` includes the query. Using the terminology of :rfc:`3986`,
+ ``path`` here includes ``hier-part`` and the ``query``.
.. attribute:: request_version
@@ -195,7 +197,7 @@ provides three different variants:
request header it responds back with a ``100 Continue`` followed by ``200
OK`` headers.
This method can be overridden to raise an error if the server does not
- want the client to continue. For e.g. server can chose to send ``417
+ want the client to continue. For e.g. server can choose to send ``417
Expectation Failed`` as a response header and ``return False``.
.. versionadded:: 3.2
@@ -318,9 +320,16 @@ provides three different variants:
.. class:: SimpleHTTPRequestHandler(request, client_address, server, directory=None)
- This class serves files from the current directory and below, directly
+ This class serves files from the directory *directory* and below,
+ or the current directory if *directory* is not provided, directly
mapping the directory structure to HTTP requests.
+ .. versionadded:: 3.7
+ The *directory* parameter.
+
+ .. versionchanged:: 3.9
+ The *directory* parameter accepts a :term:`path-like object`.
+
A lot of the work, such as parsing the request, is done by the base class
:class:`BaseHTTPRequestHandler`. This class implements the :func:`do_GET`
and :func:`do_HEAD` functions.
@@ -343,13 +352,6 @@ provides three different variants:
This dictionary is no longer filled with the default system mappings,
but only contains overrides.
- .. attribute:: directory
-
- If not specified, the directory to serve is the current working directory.
-
- .. versionchanged:: 3.9
- Accepts a :term:`path-like object`.
-
The :class:`SimpleHTTPRequestHandler` class defines the following methods:
.. method:: do_HEAD()
@@ -410,17 +412,22 @@ the current directory::
.. _http-server-cli:
:mod:`http.server` can also be invoked directly using the :option:`-m`
-switch of the interpreter with a ``port number`` argument. Similar to
+switch of the interpreter. Similar to
the previous example, this serves files relative to the current directory::
- python -m http.server 8000
+ python -m http.server
+
+The server listens to port 8000 by default. The default can be overridden
+by passing the desired port number as an argument::
-By default, server binds itself to all interfaces. The option ``-b/--bind``
+ python -m http.server 9000
+
+By default, the server binds itself to all interfaces. The option ``-b/--bind``
specifies a specific address to which it should bind. Both IPv4 and IPv6
addresses are supported. For example, the following command causes the server
to bind to localhost only::
- python -m http.server 8000 --bind 127.0.0.1
+ python -m http.server --bind 127.0.0.1
.. versionadded:: 3.4
``--bind`` argument was introduced.
@@ -428,14 +435,14 @@ to bind to localhost only::
.. versionadded:: 3.8
``--bind`` argument enhanced to support IPv6
-By default, server uses the current directory. The option ``-d/--directory``
+By default, the server uses the current directory. The option ``-d/--directory``
specifies a directory to which it should serve the files. For example,
the following command uses a specific directory::
python -m http.server --directory /tmp/
.. versionadded:: 3.7
- ``--directory`` specify alternate directory
+ ``--directory`` argument was introduced.
.. class:: CGIHTTPRequestHandler(request, client_address, server)
@@ -480,4 +487,24 @@ the following command uses a specific directory::
:class:`CGIHTTPRequestHandler` can be enabled in the command line by passing
the ``--cgi`` option::
- python -m http.server --cgi 8000
+ python -m http.server --cgi
+
+.. _http.server-security:
+
+Security Considerations
+-----------------------
+
+.. index:: pair: http.server; security
+
+:class:`SimpleHTTPRequestHandler` will follow symbolic links when handling
+requests, this makes it possible for files outside of the specified directory
+to be served.
+
+Earlier versions of Python did not scrub control characters from the
+log messages emitted to stderr from ``python -m http.server`` or the
+default :class:`BaseHTTPRequestHandler` ``.log_message``
+implementation. This could allow remote clients connecting to your
+server to send nefarious control codes to your terminal.
+
+.. versionadded:: 3.9.16
+ scrubbing control characters from log messages
diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst
index b1192e7bb46552..b8febb9a5b6213 100644
--- a/Doc/library/idle.rst
+++ b/Doc/library/idle.rst
@@ -96,11 +96,13 @@ Save Copy As...
Print Window
Print the current window to the default printer.
-Close
- Close the current window (ask to save if unsaved).
+Close Window
+ Close the current window (if an unsaved editor, ask to save; if an unsaved
+ Shell, ask to quit execution). Calling ``exit()`` or ``close()`` in the Shell
+ window also closes Shell. If this is the only window, also exit IDLE.
-Exit
- Close all windows and quit IDLE (ask to save unsaved windows).
+Exit IDLE
+ Close all windows and quit IDLE (ask to save unsaved edit windows).
Edit menu (Shell and Editor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -147,7 +149,7 @@ Go to Line
Clear any selection and update the line and column status.
Show Completions
- Open a scrollable list allowing selection of keywords and attributes. See
+ Open a scrollable list allowing selection of existing names. See
:ref:`Completions ` in the Editing and navigation section below.
Expand Word
@@ -250,7 +252,7 @@ View Last Restart
Scroll the shell window to the last Shell restart.
Restart Shell
- Restart the shell to clean the environment.
+ Restart the shell to clean the environment and reset display and exception handling.
Previous History
Cycle through earlier commands in history which match the current entry.
@@ -469,82 +471,91 @@ are restricted to four spaces due to Tcl/Tk limitations.
See also the indent/dedent region commands on the
:ref:`Format menu `.
-
.. _completions:
Completions
^^^^^^^^^^^
-Completions are supplied for functions, classes, and attributes of classes,
-both built-in and user-defined. Completions are also provided for
-filenames.
-
-The AutoCompleteWindow (ACW) will open after a predefined delay (default is
-two seconds) after a '.' or (in a string) an os.sep is typed. If after one
-of those characters (plus zero or more other characters) a tab is typed
-the ACW will open immediately if a possible continuation is found.
-
-If there is only one possible completion for the characters entered, a
-:kbd:`Tab` will supply that completion without opening the ACW.
-
-'Show Completions' will force open a completions window, by default the
-:kbd:`C-space` will open a completions window. In an empty
-string, this will contain the files in the current directory. On a
-blank line, it will contain the built-in and user-defined functions and
-classes in the current namespaces, plus any modules imported. If some
-characters have been entered, the ACW will attempt to be more specific.
-
-If a string of characters is typed, the ACW selection will jump to the
-entry most closely matching those characters. Entering a :kbd:`tab` will
-cause the longest non-ambiguous match to be entered in the Editor window or
-Shell. Two :kbd:`tab` in a row will supply the current ACW selection, as
-will return or a double click. Cursor keys, Page Up/Down, mouse selection,
-and the scroll wheel all operate on the ACW.
-
-"Hidden" attributes can be accessed by typing the beginning of hidden
-name after a '.', e.g. '_'. This allows access to modules with
-``__all__`` set, or to class-private attributes.
-
-Completions and the 'Expand Word' facility can save a lot of typing!
-
-Completions are currently limited to those in the namespaces. Names in
-an Editor window which are not via ``__main__`` and :data:`sys.modules` will
-not be found. Run the module once with your imports to correct this situation.
-Note that IDLE itself places quite a few modules in sys.modules, so
-much can be found by default, e.g. the re module.
-
-If you don't like the ACW popping up unbidden, simply make the delay
-longer or disable the extension.
+Completions are supplied, when requested and available, for module
+names, attributes of classes or functions, or filenames. Each request
+method displays a completion box with existing names. (See tab
+completions below for an exception.) For any box, change the name
+being completed and the item highlighted in the box by
+typing and deleting characters; by hitting :kbd:`Up`, :kbd:`Down`,
+:kbd:`PageUp`, :kbd:`PageDown`, :kbd:`Home`, and :kbd:`End` keys;
+and by a single click within the box. Close the box with :kbd:`Escape`,
+:kbd:`Enter`, and double :kbd:`Tab` keys or clicks outside the box.
+A double click within the box selects and closes.
+
+One way to open a box is to type a key character and wait for a
+predefined interval. This defaults to 2 seconds; customize it
+in the settings dialog. (To prevent auto popups, set the delay to a
+large number of milliseconds, such as 100000000.) For imported module
+names or class or function attributes, type '.'.
+For filenames in the root directory, type :data:`os.sep` or
+:data:`os.altsep` immediately after an opening quote. (On Windows,
+one can specify a drive first.) Move into subdirectories by typing a
+directory name and a separator.
+
+Instead of waiting, or after a box is closed, open a completion box
+immediately with Show Completions on the Edit menu. The default hot
+key is :kbd:`C-space`. If one types a prefix for the desired name
+before opening the box, the first match or near miss is made visible.
+The result is the same as if one enters a prefix
+after the box is displayed. Show Completions after a quote completes
+filenames in the current directory instead of a root directory.
+
+Hitting :kbd:`Tab` after a prefix usually has the same effect as Show
+Completions. (With no prefix, it indents.) However, if there is only
+one match to the prefix, that match is immediately added to the editor
+text without opening a box.
+
+Invoking 'Show Completions', or hitting :kbd:`Tab` after a prefix,
+outside of a string and without a preceding '.' opens a box with
+keywords, builtin names, and available module-level names.
+
+When editing code in an editor (as oppose to Shell), increase the
+available module-level names by running your code
+and not restarting the Shell thereafter. This is especially useful
+after adding imports at the top of a file. This also increases
+possible attribute completions.
+
+Completion boxes initially exclude names beginning with '_' or, for
+modules, not included in '__all__'. The hidden names can be accessed
+by typing '_' after '.', either before or after the box is opened.
.. _calltips:
Calltips
^^^^^^^^
-A calltip is shown when one types :kbd:`(` after the name of an *accessible*
-function. A name expression may include dots and subscripts. A calltip
-remains until it is clicked, the cursor is moved out of the argument area,
-or :kbd:`)` is typed. When the cursor is in the argument part of a definition,
-the menu or shortcut display a calltip.
+A calltip is shown automatically when one types :kbd:`(` after the name
+of an *accessible* function. A function name expression may include
+dots and subscripts. A calltip remains until it is clicked, the cursor
+is moved out of the argument area, or :kbd:`)` is typed. Whenever the
+cursor is in the argument part of a definition, select Edit and "Show
+Call Tip" on the menu or enter its shortcut to display a calltip.
-A calltip consists of the function signature and the first line of the
-docstring. For builtins without an accessible signature, the calltip
-consists of all lines up the fifth line or the first blank line. These
-details may change.
+The calltip consists of the function's signature and docstring up to
+the latter's first blank line or the fifth non-blank line. (Some builtin
+functions lack an accessible signature.) A '/' or '*' in the signature
+indicates that the preceding or following arguments are passed by
+position or name (keyword) only. Details are subject to change.
-The set of *accessible* functions depends on what modules have been imported
-into the user process, including those imported by Idle itself,
-and what definitions have been run, all since the last restart.
+In Shell, the accessible functions depends on what modules have been
+imported into the user process, including those imported by Idle itself,
+and which definitions have been run, all since the last restart.
For example, restart the Shell and enter ``itertools.count(``. A calltip
-appears because Idle imports itertools into the user process for its own use.
-(This could change.) Enter ``turtle.write(`` and nothing appears. Idle does
-not import turtle. The menu or shortcut do nothing either. Enter
-``import turtle`` and then ``turtle.write(`` will work.
+appears because Idle imports itertools into the user process for its own
+use. (This could change.) Enter ``turtle.write(`` and nothing appears.
+Idle does not itself import turtle. The menu entry and shortcut also do
+nothing. Enter ``import turtle``. Thereafter, ``turtle.write(``
+will display a calltip.
-In an editor, import statements have no effect until one runs the file. One
-might want to run a file after writing the import statements at the top,
-or immediately run an existing file before editing.
+In an editor, import statements have no effect until one runs the file.
+One might want to run a file after writing import statements, after
+adding function definitions, or after opening an existing file.
.. _code-context:
@@ -661,8 +672,16 @@ IDLE uses a socket to communicate between the IDLE GUI process and the user
code execution process. A connection must be established whenever the Shell
starts or restarts. (The latter is indicated by a divider line that says
'RESTART'). If the user process fails to connect to the GUI process, it
-displays a ``Tk`` error box with a 'cannot connect' message that directs the
-user here. It then exits.
+usually displays a ``Tk`` error box with a 'cannot connect' message
+that directs the user here. It then exits.
+
+One specific connection failure on Unix systems results from
+misconfigured masquerading rules somewhere in a system's network setup.
+When IDLE is started from a terminal, one will see a message starting
+with ``** Invalid host:``.
+The valid value is ``127.0.0.1 (idlelib.rpc.LOCALHOST)``.
+One can diagnose with ``tcpconnect -irv 127.0.0.1 6543`` in one
+terminal window and ``tcplisten `` in another.
A common cause of failure is a user-written file with the same name as a
standard library module, such as *random.py* and *tkinter.py*. When such a
@@ -700,6 +719,13 @@ If IDLE quits with no message, and it was not started from a console, try
starting it from a console or terminal (``python -m idlelib``) and see if
this results in an error message.
+On Unix-based systems with tcl/tk older than ``8.6.11`` (see
+``About IDLE``) certain characters of certain fonts can cause
+a tk failure with a message to the terminal. This can happen either
+if one starts IDLE to edit a file with such a character or later
+when entering such a character. If one cannot upgrade tcl/tk,
+then re-configure IDLE to use a font that works better.
+
Running user code
^^^^^^^^^^^^^^^^^
@@ -708,7 +734,7 @@ intended to be the same as executing the same code by the default method,
directly with Python in a text-mode system console or terminal window.
However, the different interface and operation occasionally affect
visible results. For instance, ``sys.modules`` starts with more entries,
-and ``threading.activeCount()`` returns 2 instead of 1.
+and ``threading.active_count()`` returns 2 instead of 1.
By default, IDLE runs user code in a separate OS process rather than in
the user interface process that runs the shell and editor. In the execution
@@ -717,28 +743,38 @@ with objects that get input from and send output to the Shell window.
The original values stored in ``sys.__stdin__``, ``sys.__stdout__``, and
``sys.__stderr__`` are not touched, but may be ``None``.
-When Shell has the focus, it controls the keyboard and screen. This is
-normally transparent, but functions that directly access the keyboard
-and screen will not work. These include system-specific functions that
-determine whether a key has been pressed and if so, which.
+Sending print output from one process to a text widget in another is
+slower than printing to a system terminal in the same process.
+This has the most effect when printing multiple arguments, as the string
+for each argument, each separator, the newline are sent separately.
+For development, this is usually not a problem, but if one wants to
+print faster in IDLE, format and join together everything one wants
+displayed together and then print a single string. Both format strings
+and :meth:`str.join` can help combine fields and lines.
IDLE's standard stream replacements are not inherited by subprocesses
-created in the execution process, whether directly by user code or by modules
-such as multiprocessing. If such subprocess use ``input`` from sys.stdin
-or ``print`` or ``write`` to sys.stdout or sys.stderr,
+created in the execution process, whether directly by user code or by
+modules such as multiprocessing. If such subprocess use ``input`` from
+sys.stdin or ``print`` or ``write`` to sys.stdout or sys.stderr,
IDLE should be started in a command line window. The secondary subprocess
will then be attached to that window for input and output.
-The IDLE code running in the execution process adds frames to the call stack
-that would not be there otherwise. IDLE wraps ``sys.getrecursionlimit`` and
-``sys.setrecursionlimit`` to reduce the effect of the additional stack frames.
-
If ``sys`` is reset by user code, such as with ``importlib.reload(sys)``,
IDLE's changes are lost and input from the keyboard and output to the screen
will not work correctly.
-When user code raises SystemExit either directly or by calling sys.exit, IDLE
-returns to a Shell prompt instead of exiting.
+When Shell has the focus, it controls the keyboard and screen. This is
+normally transparent, but functions that directly access the keyboard
+and screen will not work. These include system-specific functions that
+determine whether a key has been pressed and if so, which.
+
+The IDLE code running in the execution process adds frames to the call stack
+that would not be there otherwise. IDLE wraps ``sys.getrecursionlimit`` and
+``sys.setrecursionlimit`` to reduce the effect of the additional stack
+frames.
+
+When user code raises SystemExit either directly or by calling sys.exit,
+IDLE returns to a Shell prompt instead of exiting.
User output in Shell
^^^^^^^^^^^^^^^^^^^^
diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst
index 7c5b0750161598..65681ec093598c 100644
--- a/Doc/library/imaplib.rst
+++ b/Doc/library/imaplib.rst
@@ -143,7 +143,7 @@ The following utility functions are defined:
.. function:: Int2AP(num)
- Converts an integer into a string representation using characters from the set
+ Converts an integer into a bytes representation using characters from the set
[``A`` .. ``P``].
@@ -174,9 +174,9 @@ example of usage.
.. seealso::
- Documents describing the protocol, and sources and binaries for servers
- implementing it, can all be found at the University of Washington's *IMAP
- Information Center* (https://www.washington.edu/imap/).
+ Documents describing the protocol, sources for servers
+ implementing it, by the University of Washington's IMAP Information Center
+ can all be found at (**Source Code**) https://github.com/uw-imap/imap (**Not Maintained**).
.. _imap4-objects:
@@ -197,7 +197,7 @@ you want to avoid having an argument string quoted (eg: the *flags* argument to
Each command returns a tuple: ``(type, [data, ...])`` where *type* is usually
``'OK'`` or ``'NO'``, and *data* is either the text from the command response,
-or mandated results from the command. Each *data* is either a string, or a
+or mandated results from the command. Each *data* is either a ``bytes``, or a
tuple. If a tuple, then the first part is the header of the response, and the
second part contains the data (ie: 'literal' value).
diff --git a/Doc/library/imghdr.rst b/Doc/library/imghdr.rst
index 800e919599d9cb..186480be3e97ac 100644
--- a/Doc/library/imghdr.rst
+++ b/Doc/library/imghdr.rst
@@ -3,9 +3,14 @@
.. module:: imghdr
:synopsis: Determine the type of image contained in a file or byte stream.
+ :deprecated:
**Source code:** :source:`Lib/imghdr.py`
+.. deprecated:: 3.11
+ The :mod:`imghdr` module is deprecated
+ (see :pep:`PEP 594 <594#imghdr>` for details and alternatives).
+
--------------
The :mod:`imghdr` module determines the type of image contained in a file or
diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
index f5ad8c7229644f..121a730e0c9b4a 100644
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -8,7 +8,7 @@
**Source code:** :source:`Lib/imp.py`
.. deprecated:: 3.4
- The :mod:`imp` package is pending deprecation in favor of :mod:`importlib`.
+ The :mod:`imp` module is deprecated in favor of :mod:`importlib`.
.. index:: statement: import
diff --git a/Doc/library/importlib.metadata.rst b/Doc/library/importlib.metadata.rst
index 15e58b860d97d0..7652d501d9d917 100644
--- a/Doc/library/importlib.metadata.rst
+++ b/Doc/library/importlib.metadata.rst
@@ -4,6 +4,13 @@
Using :mod:`!importlib.metadata`
=================================
+.. module:: importlib.metadata
+ :synopsis: The implementation of the importlib metadata.
+
+**Source code:** :source:`Lib/importlib/metadata.py`
+
+.. versionadded:: 3.8
+
.. note::
This functionality is provisional and may deviate from the usual
version semantics of the standard library.
@@ -77,7 +84,9 @@ Entry points
The ``entry_points()`` function returns a dictionary of all entry points,
keyed by group. Entry points are represented by ``EntryPoint`` instances;
each ``EntryPoint`` has a ``.name``, ``.group``, and ``.value`` attributes and
-a ``.load()`` method to resolve the value.
+a ``.load()`` method to resolve the value. There are also ``.module``,
+``.attr``, and ``.extras`` attributes for getting the components of the
+``.value`` attribute::
>>> eps = entry_points() # doctest: +SKIP
>>> list(eps) # doctest: +SKIP
@@ -86,6 +95,12 @@ a ``.load()`` method to resolve the value.
>>> wheel = [ep for ep in scripts if ep.name == 'wheel'][0] # doctest: +SKIP
>>> wheel # doctest: +SKIP
EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts')
+ >>> wheel.module # doctest: +SKIP
+ 'wheel.cli'
+ >>> wheel.attr # doctest: +SKIP
+ 'main'
+ >>> wheel.extras # doctest: +SKIP
+ []
>>> main = wheel.load() # doctest: +SKIP
>>> main # doctest: +SKIP
@@ -94,7 +109,7 @@ The ``group`` and ``name`` are arbitrary values defined by the package author
and usually a client will wish to resolve all entry points for a particular
group. Read `the setuptools docs
`_
-for more information on entrypoints, their definition, and usage.
+for more information on entry points, their definition, and usage.
.. _metadata:
@@ -134,7 +149,7 @@ Distribution files
You can also get the full set of files contained within a distribution. The
``files()`` function takes a distribution package name and returns all of the
files installed by this distribution. Each file object returned is a
-``PackagePath``, a :class:`pathlib.Path` derived object with additional ``dist``,
+``PackagePath``, a :class:`pathlib.PurePath` derived object with additional ``dist``,
``size``, and ``hash`` properties as indicated by the metadata. For example::
>>> util = [p for p in files('wheel') if 'util.py' in str(p)][0] # doctest: +SKIP
@@ -158,6 +173,12 @@ Once you have the file, you can also read its contents::
return s.encode('utf-8')
return s
+You can also use the ``locate`` method to get a the absolute path to the
+file::
+
+ >>> util.locate() # doctest: +SKIP
+ PosixPath('/home/gustav/example/lib/site-packages/wheel/util.py')
+
In the case where the metadata file listing files
(RECORD or SOURCES.txt) is missing, ``files()`` will
return ``None``. The caller may wish to wrap calls to
@@ -178,6 +199,8 @@ function::
["pytest (>=3.0.0) ; extra == 'test'", "pytest-cov ; extra == 'test'"]
+.. _distributions:
+
Distributions
=============
@@ -198,9 +221,9 @@ Thus, an alternative way to get the version number is through the
There are all kinds of additional metadata available on the ``Distribution``
instance::
- >>> d.metadata['Requires-Python'] # doctest: +SKIP
+ >>> dist.metadata['Requires-Python'] # doctest: +SKIP
'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'
- >>> d.metadata['License'] # doctest: +SKIP
+ >>> dist.metadata['License'] # doctest: +SKIP
'MIT'
The full set of available metadata is not described here. See :pep:`566`
@@ -235,7 +258,7 @@ method::
"""
The ``DistributionFinder.Context`` object provides ``.path`` and ``.name``
-properties indicating the path to search and names to match and may
+properties indicating the path to search and name to match and may
supply other relevant context.
What this means in practice is that to support finding distribution package
diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
index 99bfeacbbc7407..97564815310136 100644
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -438,8 +438,9 @@ ABC hierarchy::
package. This attribute is not set on modules.
- :attr:`__package__`
- The parent package for the module/package. If the module is
- top-level then it has a value of the empty string. The
+ The fully-qualified name of the package under which the module was
+ loaded as a submodule (or the empty string for top-level modules).
+ For packages, it is the same as :attr:`__name__`. The
:func:`importlib.util.module_for_loader` decorator can handle the
details for :attr:`__package__`.
@@ -480,7 +481,7 @@ ABC hierarchy::
.. class:: ResourceReader
- *Superseded by TraversableReader*
+ *Superseded by TraversableResources*
An :term:`abstract base class` to provide the ability to read
*resources*.
@@ -611,7 +612,7 @@ ABC hierarchy::
.. method:: is_package(fullname)
- An abstract method to return a true value if the module is a package, a
+ An optional method to return a true value if the module is a package, a
false value otherwise. :exc:`ImportError` is raised if the
:term:`loader` cannot find the module.
@@ -804,8 +805,50 @@ ABC hierarchy::
.. versionadded:: 3.9
+ .. abstractmethod:: name()
-.. class:: TraversableReader
+ The base name of this object without any parent references.
+
+ .. abstractmethod:: iterdir()
+
+ Yield Traversable objects in self.
+
+ .. abstractmethod:: is_dir()
+
+ Return True if self is a directory.
+
+ .. abstractmethod:: is_file()
+
+ Return True if self is a file.
+
+ .. abstractmethod:: joinpath(child)
+
+ Return Traversable child in self.
+
+ .. abstractmethod:: __truediv__(child)
+
+ Return Traversable child in self.
+
+ .. abstractmethod:: open(mode='r', *args, **kwargs)
+
+ *mode* may be 'r' or 'rb' to open as text or binary. Return a handle
+ suitable for reading (same as :attr:`pathlib.Path.open`).
+
+ When opening as text, accepts encoding parameters such as those
+ accepted by :attr:`io.TextIOWrapper`.
+
+ .. method:: read_bytes()
+
+ Read contents of self as bytes.
+
+ .. method:: read_text(encoding=None)
+
+ Read contents of self as text.
+
+ Note: In Python 3.11 and later, this class is found in ``importlib.resources.abc``.
+
+
+.. class:: TraversableResources
An abstract base class for resource readers capable of serving
the ``files`` interface. Subclasses ResourceReader and provides
@@ -813,11 +856,10 @@ ABC hierarchy::
methods. Therefore, any loader supplying TraversableReader
also supplies ResourceReader.
- Loaders that wish to support resource reading are expected to
- implement this interface.
-
.. versionadded:: 3.9
+ Note: In Python 3.11 and later, this class is found in ``importlib.resources.abc``.
+
:mod:`importlib.resources` -- Resources
---------------------------------------
@@ -880,7 +922,7 @@ The following functions are available.
.. function:: files(package)
- Returns an :class:`importlib.resources.abc.Traversable` object
+ Returns an :class:`importlib.abc.Traversable` object
representing the resource container for the package (think directory)
and its resources (think files). A Traversable may contain other
containers (think subdirectories).
@@ -890,6 +932,22 @@ The following functions are available.
.. versionadded:: 3.9
+.. function:: as_file(traversable)
+
+ Given a :class:`importlib.abc.Traversable` object representing
+ a file, typically from :func:`importlib.resources.files`, return
+ a context manager for use in a :keyword:`with` statement.
+ The context manager provides a :class:`pathlib.Path` object.
+
+ Exiting the context manager cleans up any temporary file created when the
+ resource was extracted from e.g. a zip file.
+
+ Use ``as_file`` when the Traversable methods
+ (``read_text``, etc) are insufficient and an actual file on
+ the file system is required.
+
+ .. versionadded:: 3.9
+
.. function:: open_binary(package, resource)
Open for binary reading the *resource* within *package*.
@@ -1073,7 +1131,7 @@ find and load modules.
.. class:: WindowsRegistryFinder
- :term:`Finder` for modules declared in the Windows registry. This class
+ :term:`Finder ` for modules declared in the Windows registry. This class
implements the :class:`importlib.abc.MetaPathFinder` ABC.
Only class methods are defined by this class to alleviate the need for
@@ -1088,7 +1146,7 @@ find and load modules.
.. class:: PathFinder
- A :term:`Finder` for :data:`sys.path` and package ``__path__`` attributes.
+ A :term:`Finder ` for :data:`sys.path` and package ``__path__`` attributes.
This class implements the :class:`importlib.abc.MetaPathFinder` ABC.
Only class methods are defined by this class to alleviate the need for
@@ -1137,7 +1195,7 @@ find and load modules.
directory for ``''`` (i.e. the empty string).
-.. class:: FileFinder(path, \*loader_details)
+.. class:: FileFinder(path, *loader_details)
A concrete implementation of :class:`importlib.abc.PathEntryFinder` which
caches results from the file system.
@@ -1180,7 +1238,7 @@ find and load modules.
Clear out the internal cache.
- .. classmethod:: path_hook(\*loader_details)
+ .. classmethod:: path_hook(*loader_details)
A class method which returns a closure for use on :attr:`sys.path_hooks`.
An instance of :class:`FileFinder` is returned by the closure using the
@@ -1347,8 +1405,8 @@ find and load modules.
(``__loader__``)
- The loader to use for loading. For namespace packages this should be
- set to ``None``.
+ The :term:`Loader ` that should be used when loading
+ the module. :term:`Finders ` should always set this.
.. attribute:: origin
@@ -1381,8 +1439,9 @@ find and load modules.
(``__package__``)
- (Read-only) Fully-qualified name of the package to which the module
- belongs as a submodule (or ``None``).
+ (Read-only) The fully-qualified name of the package under which the module
+ should be loaded as a submodule (or the empty string for top-level modules).
+ For packages, it is the same as :attr:`__name__`.
.. attribute:: has_location
@@ -1474,7 +1533,7 @@ an :term:`importer`.
If **name** has no leading dots, then **name** is simply returned. This
allows for usage such as
- ``importlib.util.resolve_name('sys', __package__)`` without doing a
+ ``importlib.util.resolve_name('sys', __spec__.parent)`` without doing a
check to see if the **package** argument is needed.
:exc:`ImportError` is raised if **name** is a relative module name but
diff --git a/Doc/library/index.rst b/Doc/library/index.rst
index bebf7429b0e63e..5007bacbab1e67 100644
--- a/Doc/library/index.rst
+++ b/Doc/library/index.rst
@@ -56,7 +56,6 @@ the `Python Package Index `_.
crypto.rst
allos.rst
concurrency.rst
- contextvars.rst
ipc.rst
netdata.rst
markup.rst
@@ -76,4 +75,4 @@ the `Python Package Index `_.
windows.rst
unix.rst
superseded.rst
- undoc.rst
+ security_warnings.rst
diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst
index d00a30ff004063..d1e4a9b5f30f3d 100644
--- a/Doc/library/inspect.rst
+++ b/Doc/library/inspect.rst
@@ -556,7 +556,7 @@ The Signature object represents the call signature of a callable object and its
return annotation. To retrieve a Signature object, use the :func:`signature`
function.
-.. function:: signature(callable, \*, follow_wrapped=True)
+.. function:: signature(callable, *, follow_wrapped=True)
Return a :class:`Signature` object for the given ``callable``::
@@ -597,7 +597,7 @@ function.
C provide no metadata about their arguments.
-.. class:: Signature(parameters=None, \*, return_annotation=Signature.empty)
+.. class:: Signature(parameters=None, *, return_annotation=Signature.empty)
A Signature object represents the call signature of a function and its return
annotation. For each parameter accepted by the function it stores a
@@ -668,7 +668,7 @@ function.
>>> str(new_sig)
"(a, b) -> 'new return anno'"
- .. classmethod:: Signature.from_callable(obj, \*, follow_wrapped=True)
+ .. classmethod:: Signature.from_callable(obj, *, follow_wrapped=True)
Return a :class:`Signature` (or its subclass) object for a given callable
``obj``. Pass ``follow_wrapped=False`` to get a signature of ``obj``
@@ -684,7 +684,7 @@ function.
.. versionadded:: 3.5
-.. class:: Parameter(name, kind, \*, default=Parameter.empty, annotation=Parameter.empty)
+.. class:: Parameter(name, kind, *, default=Parameter.empty, annotation=Parameter.empty)
Parameter objects are *immutable*. Instead of modifying a Parameter object,
you can use :meth:`Parameter.replace` to create a modified copy.
@@ -809,10 +809,10 @@ function.
>>> str(param.replace(default=Parameter.empty, annotation='spam'))
"foo:'spam'"
- .. versionchanged:: 3.4
- In Python 3.3 Parameter objects were allowed to have ``name`` set
- to ``None`` if their ``kind`` was set to ``POSITIONAL_ONLY``.
- This is no longer permitted.
+ .. versionchanged:: 3.4
+ In Python 3.3 Parameter objects were allowed to have ``name`` set
+ to ``None`` if their ``kind`` was set to ``POSITIONAL_ONLY``.
+ This is no longer permitted.
.. class:: BoundArguments
diff --git a/Doc/library/internet.rst b/Doc/library/internet.rst
index b8950bb6cb8c2d..c8bab2a5836ade 100644
--- a/Doc/library/internet.rst
+++ b/Doc/library/internet.rst
@@ -20,8 +20,6 @@ is currently supported on most popular platforms. Here is an overview:
.. toctree::
webbrowser.rst
- cgi.rst
- cgitb.rst
wsgiref.rst
urllib.rst
urllib.request.rst
@@ -33,10 +31,7 @@ is currently supported on most popular platforms. Here is an overview:
ftplib.rst
poplib.rst
imaplib.rst
- nntplib.rst
smtplib.rst
- smtpd.rst
- telnetlib.rst
uuid.rst
socketserver.rst
http.server.rst
diff --git a/Doc/library/intro.rst b/Doc/library/intro.rst
index 8567e4d1d08920..5bb33b9c10cc03 100644
--- a/Doc/library/intro.rst
+++ b/Doc/library/intro.rst
@@ -58,5 +58,5 @@ Notes on availability
operating system.
* If not separately noted, all functions that claim "Availability: Unix" are
- supported on Mac OS X, which builds on a Unix core.
+ supported on macOS, which builds on a Unix core.
diff --git a/Doc/library/io.rst b/Doc/library/io.rst
index aecbec56866d73..ea57aae71a2352 100644
--- a/Doc/library/io.rst
+++ b/Doc/library/io.rst
@@ -155,16 +155,6 @@ High-level Module Interface
when an unsupported operation is called on a stream.
-In-memory streams
-^^^^^^^^^^^^^^^^^
-
-It is also possible to use a :class:`str` or :term:`bytes-like object` as a
-file for both reading and writing. For strings :class:`StringIO` can be used
-like a file opened in text mode. :class:`BytesIO` can be used like a file
-opened in binary mode. Both provide full read-write capabilities with random
-access.
-
-
.. seealso::
:mod:`sys`
@@ -240,8 +230,7 @@ I/O Base Classes
.. class:: IOBase
- The abstract base class for all I/O classes, acting on streams of bytes.
- There is no public constructor.
+ The abstract base class for all I/O classes.
This class provides empty abstract implementations for many methods
that derived classes can override selectively; the default
@@ -310,7 +299,7 @@ I/O Base Classes
Return ``True`` if the stream can be read from. If ``False``, :meth:`read`
will raise :exc:`OSError`.
- .. method:: readline(size=-1)
+ .. method:: readline(size=-1, /)
Read and return one line from the stream. If *size* is specified, at
most *size* bytes will be read.
@@ -319,16 +308,19 @@ I/O Base Classes
the *newline* argument to :func:`open` can be used to select the line
terminator(s) recognized.
- .. method:: readlines(hint=-1)
+ .. method:: readlines(hint=-1, /)
Read and return a list of lines from the stream. *hint* can be specified
to control the number of lines read: no more lines will be read if the
total size (in bytes/characters) of all lines so far exceeds *hint*.
+ *hint* values of ``0`` or less, as well as ``None``, are treated as no
+ hint.
+
Note that it's already possible to iterate on file objects using ``for
line in file: ...`` without calling ``file.readlines()``.
- .. method:: seek(offset, whence=SEEK_SET)
+ .. method:: seek(offset, whence=SEEK_SET, /)
Change the stream position to the given byte *offset*. *offset* is
interpreted relative to the position indicated by *whence*. The default
@@ -360,7 +352,7 @@ I/O Base Classes
Return the current stream position.
- .. method:: truncate(size=None)
+ .. method:: truncate(size=None, /)
Resize the stream to the given *size* in bytes (or the current position
if *size* is not specified). The current stream position isn't changed.
@@ -377,7 +369,7 @@ I/O Base Classes
Return ``True`` if the stream supports writing. If ``False``,
:meth:`write` and :meth:`truncate` will raise :exc:`OSError`.
- .. method:: writelines(lines)
+ .. method:: writelines(lines, /)
Write a list of lines to the stream. Line separators are not added, so it
is usual for each of the lines provided to have a line separator at the
@@ -392,8 +384,7 @@ I/O Base Classes
.. class:: RawIOBase
- Base class for raw binary streams. It inherits :class:`IOBase`. There is no
- public constructor.
+ Base class for raw binary streams. It inherits :class:`IOBase`.
Raw binary streams typically provide low-level access to an underlying OS
device or API, and do not try to encapsulate it in high-level primitives
@@ -403,7 +394,7 @@ I/O Base Classes
:class:`RawIOBase` provides these methods in addition to those from
:class:`IOBase`:
- .. method:: read(size=-1)
+ .. method:: read(size=-1, /)
Read up to *size* bytes from the object and return them. As a convenience,
if *size* is unspecified or -1, all bytes until EOF are returned.
@@ -422,7 +413,7 @@ I/O Base Classes
Read and return all the bytes from the stream until EOF, using multiple
calls to the stream if necessary.
- .. method:: readinto(b)
+ .. method:: readinto(b, /)
Read bytes into a pre-allocated, writable
:term:`bytes-like object` *b*, and return the
@@ -430,7 +421,7 @@ I/O Base Classes
If the object is in non-blocking mode and no bytes
are available, ``None`` is returned.
- .. method:: write(b)
+ .. method:: write(b, /)
Write the given :term:`bytes-like object`, *b*, to the
underlying raw stream, and return the number of
@@ -446,7 +437,7 @@ I/O Base Classes
.. class:: BufferedIOBase
Base class for binary streams that support some kind of buffering.
- It inherits :class:`IOBase`. There is no public constructor.
+ It inherits :class:`IOBase`.
The main difference with :class:`RawIOBase` is that methods :meth:`read`,
:meth:`readinto` and :meth:`write` will try (respectively) to read as much
@@ -487,7 +478,7 @@ I/O Base Classes
.. versionadded:: 3.1
- .. method:: read(size=-1)
+ .. method:: read(size=-1, /)
Read and return up to *size* bytes. If the argument is omitted, ``None``,
or negative, data is read and returned until EOF is reached. An empty
@@ -502,7 +493,7 @@ I/O Base Classes
A :exc:`BlockingIOError` is raised if the underlying raw stream is in
non blocking-mode, and has no data available at the moment.
- .. method:: read1([size])
+ .. method:: read1(size=-1, /)
Read and return up to *size* bytes, with at most one call to the
underlying raw stream's :meth:`~RawIOBase.read` (or
@@ -513,7 +504,7 @@ I/O Base Classes
If *size* is ``-1`` (the default), an arbitrary number of bytes are
returned (more than zero unless EOF is reached).
- .. method:: readinto(b)
+ .. method:: readinto(b, /)
Read bytes into a pre-allocated, writable
:term:`bytes-like object` *b* and return the number of bytes read.
@@ -525,7 +516,7 @@ I/O Base Classes
A :exc:`BlockingIOError` is raised if the underlying raw stream is in non
blocking-mode, and has no data available at the moment.
- .. method:: readinto1(b)
+ .. method:: readinto1(b, /)
Read bytes into a pre-allocated, writable
:term:`bytes-like object` *b*, using at most one call to
@@ -537,7 +528,7 @@ I/O Base Classes
.. versionadded:: 3.5
- .. method:: write(b)
+ .. method:: write(b, /)
Write the given :term:`bytes-like object`, *b*, and return the number
of bytes written (always equal to the length of *b* in bytes, since if
@@ -620,7 +611,7 @@ Buffered Streams
Buffered I/O streams provide a higher-level interface to an I/O device
than raw I/O does.
-.. class:: BytesIO([initial_bytes])
+.. class:: BytesIO(initial_bytes=b'')
A binary stream using an in-memory bytes buffer. It inherits
:class:`BufferedIOBase`. The buffer is discarded when the
@@ -655,14 +646,14 @@ than raw I/O does.
Return :class:`bytes` containing the entire contents of the buffer.
- .. method:: read1([size])
+ .. method:: read1(size=-1, /)
In :class:`BytesIO`, this is the same as :meth:`~BufferedIOBase.read`.
.. versionchanged:: 3.7
The *size* argument is now optional.
- .. method:: readinto1(b)
+ .. method:: readinto1(b, /)
In :class:`BytesIO`, this is the same as :meth:`~BufferedIOBase.readinto`.
@@ -685,18 +676,18 @@ than raw I/O does.
:class:`BufferedReader` provides or overrides these methods in addition to
those from :class:`BufferedIOBase` and :class:`IOBase`:
- .. method:: peek([size])
+ .. method:: peek(size=0, /)
Return bytes from the stream without advancing the position. At most one
single read on the raw stream is done to satisfy the call. The number of
bytes returned may be less or more than requested.
- .. method:: read([size])
+ .. method:: read(size=-1, /)
Read and return *size* bytes, or if *size* is not given or negative, until
EOF or if the read call would block in non-blocking mode.
- .. method:: read1([size])
+ .. method:: read1(size=-1, /)
Read and return up to *size* bytes with only one call on the raw stream.
If at least one byte is buffered, only buffered bytes are returned.
@@ -733,7 +724,7 @@ than raw I/O does.
Force bytes held in the buffer into the raw stream. A
:exc:`BlockingIOError` should be raised if the raw stream blocks.
- .. method:: write(b)
+ .. method:: write(b, /)
Write the :term:`bytes-like object`, *b*, and return the
number of bytes written. When in non-blocking mode, a
@@ -756,7 +747,7 @@ than raw I/O does.
are guaranteed to be implemented.
-.. class:: BufferedRWPair(reader, writer, buffer_size=DEFAULT_BUFFER_SIZE)
+.. class:: BufferedRWPair(reader, writer, buffer_size=DEFAULT_BUFFER_SIZE, /)
A buffered binary stream providing higher-level access to two non seekable
:class:`RawIOBase` raw binary streams---one readable, the other writeable.
@@ -783,8 +774,7 @@ Text I/O
.. class:: TextIOBase
Base class for text streams. This class provides a character and line based
- interface to stream I/O. It inherits :class:`IOBase`. There is no public
- constructor.
+ interface to stream I/O. It inherits :class:`IOBase`.
:class:`TextIOBase` provides or overrides these data attributes and
methods in addition to those from :class:`IOBase`:
@@ -824,19 +814,19 @@ Text I/O
.. versionadded:: 3.1
- .. method:: read(size=-1)
+ .. method:: read(size=-1, /)
Read and return at most *size* characters from the stream as a single
:class:`str`. If *size* is negative or ``None``, reads until EOF.
- .. method:: readline(size=-1)
+ .. method:: readline(size=-1, /)
Read until newline or EOF and return a single ``str``. If the stream is
already at EOF, an empty string is returned.
If *size* is specified, at most *size* characters will be read.
- .. method:: seek(offset, whence=SEEK_SET)
+ .. method:: seek(offset, whence=SEEK_SET, /)
Change the stream position to the given *offset*. Behaviour depends on
the *whence* parameter. The default value for *whence* is
@@ -863,7 +853,7 @@ Text I/O
does not usually represent a number of bytes in the underlying
binary storage.
- .. method:: write(s)
+ .. method:: write(s, /)
Write the string *s* to the stream and return the number of characters
written.
diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst
index 5f5e66412da477..f9c1ebf3f3df26 100644
--- a/Doc/library/ipaddress.rst
+++ b/Doc/library/ipaddress.rst
@@ -41,7 +41,7 @@ IP addresses, networks and interfaces:
Return an :class:`IPv4Address` or :class:`IPv6Address` object depending on
the IP address passed as argument. Either IPv4 or IPv6 addresses may be
- supplied; integers less than 2**32 will be considered to be IPv4 by default.
+ supplied; integers less than ``2**32`` will be considered to be IPv4 by default.
A :exc:`ValueError` is raised if *address* does not represent a valid IPv4
or IPv6 address.
@@ -56,7 +56,7 @@ IP addresses, networks and interfaces:
Return an :class:`IPv4Network` or :class:`IPv6Network` object depending on
the IP address passed as argument. *address* is a string or integer
representing the IP network. Either IPv4 or IPv6 networks may be supplied;
- integers less than 2**32 will be considered to be IPv4 by default. *strict*
+ integers less than ``2**32`` will be considered to be IPv4 by default. *strict*
is passed to :class:`IPv4Network` or :class:`IPv6Network` constructor. A
:exc:`ValueError` is raised if *address* does not represent a valid IPv4 or
IPv6 address, or if the network has host bits set.
@@ -70,7 +70,7 @@ IP addresses, networks and interfaces:
Return an :class:`IPv4Interface` or :class:`IPv6Interface` object depending
on the IP address passed as argument. *address* is a string or integer
representing the IP address. Either IPv4 or IPv6 addresses may be supplied;
- integers less than 2**32 will be considered to be IPv4 by default. A
+ integers less than ``2**32`` will be considered to be IPv4 by default. A
:exc:`ValueError` is raised if *address* does not represent a valid IPv4 or
IPv6 address.
@@ -104,8 +104,7 @@ write code that handles both IP versions correctly. Address objects are
1. A string in decimal-dot notation, consisting of four decimal integers in
the inclusive range 0--255, separated by dots (e.g. ``192.168.0.1``). Each
integer represents an octet (byte) in the address. Leading zeroes are
- tolerated only for values less than 8 (as there is no ambiguity
- between the decimal and octal interpretations of such strings).
+ not tolerated to prevent confusion with octal notation.
2. An integer that fits into 32 bits.
3. An integer packed into a :class:`bytes` object of length 4 (most
significant octet first).
@@ -117,6 +116,27 @@ write code that handles both IP versions correctly. Address objects are
>>> ipaddress.IPv4Address(b'\xC0\xA8\x00\x01')
IPv4Address('192.168.0.1')
+ .. versionchanged:: 3.8
+
+ Leading zeros are tolerated, even in ambiguous cases that look like
+ octal notation.
+
+ .. versionchanged:: 3.10
+
+ Leading zeros are no longer tolerated and are treated as an error.
+ IPv4 address strings are now parsed as strict as glibc
+ :func:`~socket.inet_pton`.
+
+ .. versionchanged:: 3.9.5
+
+ The above change was also included in Python 3.9 starting with
+ version 3.9.5.
+
+ .. versionchanged:: 3.8.12
+
+ The above change was also included in Python 3.8 starting with
+ version 3.8.12.
+
.. attribute:: version
The appropriate version number: ``4`` for IPv4, ``6`` for IPv6.
@@ -168,18 +188,53 @@ write code that handles both IP versions correctly. Address objects are
.. attribute:: is_private
- ``True`` if the address is allocated for private networks. See
+ ``True`` if the address is defined as not globally reachable by
iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_
- (for IPv6).
+ (for IPv6) with the following exceptions:
+
+ * ``is_private`` is ``False`` for the shared address space (``100.64.0.0/10``)
+ * For IPv4-mapped IPv6-addresses the ``is_private`` value is determined by the
+ semantics of the underlying IPv4 addresses and the following condition holds
+ (see :attr:`IPv6Address.ipv4_mapped`)::
+
+ address.is_private == address.ipv4_mapped.is_private
+
+ ``is_private`` has value opposite to :attr:`is_global`, except for the shared address space
+ (``100.64.0.0/10`` range) where they are both ``False``.
+
+ .. versionchanged:: 3.9.20
+
+ Fixed some false positives and false negatives.
+
+ * ``192.0.0.0/24`` is considered private with the exception of ``192.0.0.9/32`` and
+ ``192.0.0.10/32`` (previously: only the ``192.0.0.0/29`` sub-range was considered private).
+ * ``64:ff9b:1::/48`` is considered private.
+ * ``2002::/16`` is considered private.
+ * There are exceptions within ``2001::/23`` (otherwise considered private): ``2001:1::1/128``,
+ ``2001:1::2/128``, ``2001:3::/32``, ``2001:4:112::/48``, ``2001:20::/28``, ``2001:30::/28``.
+ The exceptions are not considered private.
.. attribute:: is_global
- ``True`` if the address is allocated for public networks. See
+ ``True`` if the address is defined as globally reachable by
iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_
- (for IPv6).
+ (for IPv6) with the following exception:
+
+ For IPv4-mapped IPv6-addresses the ``is_private`` value is determined by the
+ semantics of the underlying IPv4 addresses and the following condition holds
+ (see :attr:`IPv6Address.ipv4_mapped`)::
+
+ address.is_global == address.ipv4_mapped.is_global
+
+ ``is_global`` has value opposite to :attr:`is_private`, except for the shared address space
+ (``100.64.0.0/10`` range) where they are both ``False``.
.. versionadded:: 3.4
+ .. versionchanged:: 3.9.20
+
+ Fixed some false positives and false negatives, see :attr:`is_private` for details.
+
.. attribute:: is_unspecified
``True`` if the address is unspecified. See :RFC:`5735` (for IPv4)
@@ -202,6 +257,32 @@ write code that handles both IP versions correctly. Address objects are
.. _iana-ipv4-special-registry: https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml
.. _iana-ipv6-special-registry: https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml
+.. method:: IPv4Address.__format__(fmt)
+
+ Returns a string representation of the IP address, controlled by
+ an explicit format string.
+ *fmt* can be one of the following: ``'s'``, the default option,
+ equivalent to :func:`str`, ``'b'`` for a zero-padded binary string,
+ ``'X'`` or ``'x'`` for an uppercase or lowercase hexadecimal
+ representation, or ``'n'``, which is equivalent to ``'b'`` for IPv4
+ addresses and ``'x'`` for IPv6. For binary and hexadecimal
+ representations, the form specifier ``'#'`` and the grouping option
+ ``'_'`` are available. ``__format__`` is used by ``format``, ``str.format``
+ and f-strings.
+
+ >>> format(ipaddress.IPv4Address('192.168.0.1'))
+ '192.168.0.1'
+ >>> '{:#b}'.format(ipaddress.IPv4Address('192.168.0.1'))
+ '0b11000000101010000000000000000001'
+ >>> f'{ipaddress.IPv6Address("2001:db8::1000"):s}'
+ '2001:db8::1000'
+ >>> format(ipaddress.IPv6Address('2001:db8::1000'), '_X')
+ '2001_0DB8_0000_0000_0000_0000_0000_1000'
+ >>> '{:#_n}'.format(ipaddress.IPv6Address('2001:db8::1000'))
+ '0x2001_0db8_0000_0000_0000_0000_0000_1000'
+
+ .. versionadded:: 3.9
+
.. class:: IPv6Address(address)
@@ -246,8 +327,8 @@ write code that handles both IP versions correctly. Address objects are
groups consisting entirely of zeroes included.
- For the following attributes, see the corresponding documentation of the
- :class:`IPv4Address` class:
+ For the following attributes and methods, see the corresponding
+ documentation of the :class:`IPv4Address` class:
.. attribute:: packed
.. attribute:: reverse_pointer
@@ -297,6 +378,12 @@ write code that handles both IP versions correctly. Address objects are
the embedded ``(server, client)`` IP address pair. For any other
address, this property will be ``None``.
+.. method:: IPv6Address.__format__(fmt)
+
+ Refer to the corresponding method documentation in
+ :class:`IPv4Address`.
+
+ .. versionadded:: 3.9
Conversion to Strings and Integers
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -630,7 +717,7 @@ dictionaries.
Note that currently expanded netmasks are not supported. That means
``2001:db00::0/24`` is a valid argument while ``2001:db00::0/ffff:ff00::``
- not.
+ is not.
2. An integer that fits into 128 bits. This is equivalent to a
single-address network, with the network address being *address* and
diff --git a/Doc/library/ipc.rst b/Doc/library/ipc.rst
index b88a174eb97f15..4849c82f317d97 100644
--- a/Doc/library/ipc.rst
+++ b/Doc/library/ipc.rst
@@ -22,7 +22,5 @@ The list of modules described in this chapter is:
ssl.rst
select.rst
selectors.rst
- asyncore.rst
- asynchat.rst
signal.rst
mmap.rst
diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst
index 3e5a673898106c..6da55f8a3f49cf 100644
--- a/Doc/library/itertools.rst
+++ b/Doc/library/itertools.rst
@@ -199,9 +199,9 @@ loops that truncate the stream.
Return *r* length subsequences of elements from the input *iterable*.
- Combinations are emitted in lexicographic sort order. So, if the
- input *iterable* is sorted, the combination tuples will be produced
- in sorted order.
+ The combination tuples are emitted in lexicographic ordering according to
+ the order of the input *iterable*. So, if the input *iterable* is sorted,
+ the combination tuples will be produced in sorted order.
Elements are treated as unique based on their position, not on their
value. So if the input elements are unique, there will be no repeat
@@ -248,9 +248,9 @@ loops that truncate the stream.
Return *r* length subsequences of elements from the input *iterable*
allowing individual elements to be repeated more than once.
- Combinations are emitted in lexicographic sort order. So, if the
- input *iterable* is sorted, the combination tuples will be produced
- in sorted order.
+ The combination tuples are emitted in lexicographic ordering according to
+ the order of the input *iterable*. So, if the input *iterable* is sorted,
+ the combination tuples will be produced in sorted order.
Elements are treated as unique based on their position, not on their
value. So if the input elements are unique, the generated combinations
@@ -484,9 +484,9 @@ loops that truncate the stream.
of the *iterable* and all possible full-length permutations
are generated.
- Permutations are emitted in lexicographic sort order. So, if the
- input *iterable* is sorted, the permutation tuples will be produced
- in sorted order.
+ The permutation tuples are emitted in lexicographic ordering according to
+ the order of the input *iterable*. So, if the input *iterable* is sorted,
+ the combination tuples will be produced in sorted order.
Elements are treated as unique based on their position, not on their
value. So if the input elements are unique, there will be no repeat
@@ -563,6 +563,9 @@ loops that truncate the stream.
for prod in result:
yield tuple(prod)
+ Before :func:`product` runs, it completely consumes the input iterables,
+ keeping pools of values in memory to generate the products. Accordingly,
+ it is only useful with finite inputs.
.. function:: repeat(object[, times])
@@ -752,7 +755,7 @@ which incur interpreter overhead.
"Count how many times the predicate is true"
return sum(map(pred, iterable))
- def padnone(iterable):
+ def pad_none(iterable):
"""Returns the sequence elements and then returns None indefinitely.
Useful for emulating the behavior of the built-in map() function.
@@ -766,6 +769,18 @@ which incur interpreter overhead.
def dotproduct(vec1, vec2):
return sum(map(operator.mul, vec1, vec2))
+ def convolve(signal, kernel):
+ # See: https://betterexplained.com/articles/intuitive-convolution/
+ # convolve(data, [0.25, 0.25, 0.25, 0.25]) --> Moving average (blur)
+ # convolve(data, [1, -1]) --> 1st finite difference (1st derivative)
+ # convolve(data, [1, -2, 1]) --> 2nd finite difference (2nd derivative)
+ kernel = tuple(kernel)[::-1]
+ n = len(kernel)
+ window = collections.deque([0], maxlen=n) * n
+ for x in chain(signal, repeat(0, n-1)):
+ window.append(x)
+ yield sum(map(operator.mul, kernel, window))
+
def flatten(list_of_lists):
"Flatten one level of nesting"
return chain.from_iterable(list_of_lists)
@@ -806,7 +821,7 @@ which incur interpreter overhead.
nexts = cycle(islice(nexts, num_active))
def partition(pred, iterable):
- 'Use a predicate to partition entries into false entries and true entries'
+ "Use a predicate to partition entries into false entries and true entries"
# partition(is_odd, range(10)) --> 0 2 4 6 8 and 1 3 5 7 9
t1, t2 = tee(iterable)
return filterfalse(pred, t1), filter(pred, t2)
@@ -878,7 +893,7 @@ which incur interpreter overhead.
def random_product(*args, repeat=1):
"Random selection from itertools.product(*args, **kwds)"
pools = [tuple(pool) for pool in args] * repeat
- return tuple(random.choice(pool) for pool in pools)
+ return tuple(map(random.choice, pools))
def random_permutation(iterable, r=None):
"Random selection from itertools.permutations(iterable, r)"
@@ -897,11 +912,11 @@ which incur interpreter overhead.
"Random selection from itertools.combinations_with_replacement(iterable, r)"
pool = tuple(iterable)
n = len(pool)
- indices = sorted(random.randrange(n) for i in range(r))
+ indices = sorted(random.choices(range(n), k=r))
return tuple(pool[i] for i in indices)
def nth_combination(iterable, r, index):
- 'Equivalent to list(combinations(iterable, r))[index]'
+ "Equivalent to list(combinations(iterable, r))[index]"
pool = tuple(iterable)
n = len(pool)
if r < 0 or r > n:
diff --git a/Doc/library/json.rst b/Doc/library/json.rst
index b923c8e1670529..4dc085fda8d385 100644
--- a/Doc/library/json.rst
+++ b/Doc/library/json.rst
@@ -11,13 +11,18 @@
--------------
-`JSON (JavaScript Object Notation) `_, specified by
+`JSON (JavaScript Object Notation) `_, specified by
:rfc:`7159` (which obsoletes :rfc:`4627`) and by
-`ECMA-404 `_,
+`ECMA-404 `_ object literal syntax
(although it is not a strict subset of JavaScript [#rfc-errata]_ ).
+.. warning::
+ Be cautious when parsing JSON data from untrusted sources. A malicious
+ JSON string may cause the decoder to consume considerable CPU and memory
+ resources. Limiting the size of data to be parsed is recommended.
+
:mod:`json` exposes an API familiar to users of the standard library
:mod:`marshal` and :mod:`pickle` modules.
@@ -125,13 +130,6 @@ See :ref:`json-commandline` for detailed documentation.
This module's encoders and decoders preserve input and output order by
default. Order is only lost if the underlying containers are unordered.
- Prior to Python 3.7, :class:`dict` was not guaranteed to be ordered, so
- inputs and outputs were typically scrambled unless
- :class:`collections.OrderedDict` was specifically requested. Starting
- with Python 3.7, the regular :class:`dict` became order preserving, so
- it is no longer necessary to specify :class:`collections.OrderedDict` for
- JSON generation and parsing.
-
Basic Usage
-----------
@@ -159,7 +157,7 @@ Basic Usage
If *check_circular* is false (default: ``True``), then the circular
reference check for container types will be skipped and a circular reference
- will result in an :exc:`OverflowError` (or worse).
+ will result in an :exc:`RecursionError` (or worse).
If *allow_nan* is false (default: ``True``), then it will be a
:exc:`ValueError` to serialize out of range :class:`float` values (``nan``,
@@ -255,6 +253,12 @@ Basic Usage
be used to use another datatype or parser for JSON integers
(e.g. :class:`float`).
+ .. versionchanged:: 3.9.14
+ The default *parse_int* of :func:`int` now limits the maximum length of
+ the integer string via the interpreter's :ref:`integer string
+ conversion length limitation ` to help avoid denial
+ of service attacks.
+
*parse_constant*, if specified, will be called with one of the following
strings: ``'-Infinity'``, ``'Infinity'``, ``'NaN'``.
This can be used to raise an exception if invalid JSON numbers
@@ -333,7 +337,7 @@ Encoders and Decoders
*object_hook*, if specified, will be called with the result of every JSON
object decoded and its return value will be used in place of the given
:class:`dict`. This can be used to provide custom deserializations (e.g. to
- support JSON-RPC class hinting).
+ support `JSON-RPC `_ class hinting).
*object_pairs_hook*, if specified will be called with the result of every
JSON object decoded with an ordered list of pairs. The return value of
@@ -422,10 +426,9 @@ Encoders and Decoders
for ``o`` if possible, otherwise it should call the superclass implementation
(to raise :exc:`TypeError`).
- If *skipkeys* is false (the default), then it is a :exc:`TypeError` to
- attempt encoding of keys that are not :class:`str`, :class:`int`,
- :class:`float` or ``None``. If *skipkeys* is true, such items are simply
- skipped.
+ If *skipkeys* is false (the default), a :exc:`TypeError` will be raised when
+ trying to encode keys that are not :class:`str`, :class:`int`, :class:`float`
+ or ``None``. If *skipkeys* is true, such items are simply skipped.
If *ensure_ascii* is true (the default), the output is guaranteed to
have all incoming non-ASCII characters escaped. If *ensure_ascii* is
@@ -433,7 +436,7 @@ Encoders and Decoders
If *check_circular* is true (the default), then lists, dicts, and custom
encoded objects will be checked for circular references during encoding to
- prevent an infinite recursion (which would cause an :exc:`OverflowError`).
+ prevent an infinite recursion (which would cause an :exc:`RecursionError`).
Otherwise, no such check takes place.
If *allow_nan* is true (the default), then ``NaN``, ``Infinity``, and
@@ -479,8 +482,8 @@ Encoders and Decoders
object for *o*, or calls the base implementation (to raise a
:exc:`TypeError`).
- For example, to support arbitrary iterators, you could implement default
- like this::
+ For example, to support arbitrary iterators, you could implement
+ :meth:`default` like this::
def default(self, o):
try:
@@ -545,7 +548,7 @@ Standard Compliance and Interoperability
----------------------------------------
The JSON format is specified by :rfc:`7159` and by
-`ECMA-404