Skip to content

Commit 5e34b30

Browse files
committed
Deploying to gh-pages from @ e063e01 🚀
1 parent 1d8711a commit 5e34b30

File tree

565 files changed

+780
-771
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

565 files changed

+780
-771
lines changed

_sources/howto/logging-cookbook.rst.txt

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4065,6 +4065,68 @@ lines. With this approach, you get better output:
40654065
WARNING:demo: 1/0
40664066
WARNING:demo:ZeroDivisionError: division by zero
40674067
4068+
How to uniformly handle newlines in logging output
4069+
--------------------------------------------------
4070+
4071+
Usually, messages that are logged (say to console or file) consist of a single
4072+
line of text. However, sometimes there is a need to handle messages with
4073+
multiple lines - whether because a logging format string contains newlines, or
4074+
logged data contains newlines. If you want to handle such messages uniformly, so
4075+
that each line in the logged message appears uniformly formatted as if it was
4076+
logged separately, you can do this using a handler mixin, as in the following
4077+
snippet:
4078+
4079+
.. code-block:: python
4080+
4081+
# Assume this is in a module mymixins.py
4082+
import copy
4083+
4084+
class MultilineMixin:
4085+
def emit(self, record):
4086+
s = record.getMessage()
4087+
if '\n' not in s:
4088+
super().emit(record)
4089+
else:
4090+
lines = s.splitlines()
4091+
rec = copy.copy(record)
4092+
rec.args = None
4093+
for line in lines:
4094+
rec.msg = line
4095+
super().emit(rec)
4096+
4097+
You can use the mixin as in the following script:
4098+
4099+
.. code-block:: python
4100+
4101+
import logging
4102+
4103+
from mymixins import MultilineMixin
4104+
4105+
logger = logging.getLogger(__name__)
4106+
4107+
class StreamHandler(MultilineMixin, logging.StreamHandler):
4108+
pass
4109+
4110+
if __name__ == '__main__':
4111+
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(levelname)-9s %(message)s',
4112+
handlers = [StreamHandler()])
4113+
logger.debug('Single line')
4114+
logger.debug('Multiple lines:\nfool me once ...')
4115+
logger.debug('Another single line')
4116+
logger.debug('Multiple lines:\n%s', 'fool me ...\ncan\'t get fooled again')
4117+
4118+
The script, when run, prints something like:
4119+
4120+
.. code-block:: text
4121+
4122+
2025-07-02 13:54:47,234 DEBUG Single line
4123+
2025-07-02 13:54:47,234 DEBUG Multiple lines:
4124+
2025-07-02 13:54:47,234 DEBUG fool me once ...
4125+
2025-07-02 13:54:47,234 DEBUG Another single line
4126+
2025-07-02 13:54:47,234 DEBUG Multiple lines:
4127+
2025-07-02 13:54:47,234 DEBUG fool me ...
4128+
2025-07-02 13:54:47,234 DEBUG can't get fooled again
4129+
40684130
40694131
.. patterns-to-avoid:
40704132

_sources/library/pyexpat.rst.txt

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,11 +16,10 @@
1616
references to these attributes should be marked using the :member: role.
1717
1818
19-
.. warning::
19+
.. note::
2020

21-
The :mod:`pyexpat` module is not secure against maliciously
22-
constructed data. If you need to parse untrusted or unauthenticated data see
23-
:ref:`xml-vulnerabilities`.
21+
If you need to parse untrusted or unauthenticated data, see
22+
:ref:`xml-security`.
2423

2524

2625
.. index:: single: Expat

_sources/library/random.rst.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -447,6 +447,11 @@ Alternative Generator
447447
Override this method in subclasses to customise the
448448
:meth:`~random.getrandbits` behaviour of :class:`!Random` instances.
449449

450+
.. method:: Random.randbytes(n)
451+
452+
Override this method in subclasses to customise the
453+
:meth:`~random.randbytes` behaviour of :class:`!Random` instances.
454+
450455

451456
.. class:: SystemRandom([seed])
452457

_sources/library/security_warnings.rst.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ The following modules have specific security considerations:
2828
<subprocess-security>`
2929
* :mod:`tempfile`: :ref:`mktemp is deprecated due to vulnerability to race
3030
conditions <tempfile-mktemp-deprecated>`
31-
* :mod:`xml`: :ref:`XML vulnerabilities <xml-vulnerabilities>`
31+
* :mod:`xml`: :ref:`XML security <xml-security>`
3232
* :mod:`zipfile`: :ref:`maliciously prepared .zip files can cause disk volume
3333
exhaustion <zipfile-resources-limitations>`
3434

_sources/library/xml.dom.minidom.rst.txt

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,10 @@ not already proficient with the DOM should consider using the
1919
:mod:`xml.etree.ElementTree` module for their XML processing instead.
2020

2121

22-
.. warning::
22+
.. note::
2323

24-
The :mod:`xml.dom.minidom` module is not secure against
25-
maliciously constructed data. If you need to parse untrusted or
26-
unauthenticated data see :ref:`xml-vulnerabilities`.
24+
If you need to parse untrusted or unauthenticated data, see
25+
:ref:`xml-security`.
2726

2827

2928
DOM applications typically start by parsing some XML into a DOM. With

_sources/library/xml.dom.pulldom.rst.txt

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,10 @@ responsible for explicitly pulling events from the stream, looping over those
1919
events until either processing is finished or an error condition occurs.
2020

2121

22-
.. warning::
22+
.. note::
2323

24-
The :mod:`xml.dom.pulldom` module is not secure against
25-
maliciously constructed data. If you need to parse untrusted or
26-
unauthenticated data see :ref:`xml-vulnerabilities`.
24+
If you need to parse untrusted or unauthenticated data, see
25+
:ref:`xml-security`.
2726

2827
.. versionchanged:: 3.7.1
2928

_sources/library/xml.etree.elementtree.rst.txt

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,10 @@ for parsing and creating XML data.
2020
The :mod:`!xml.etree.cElementTree` module is deprecated.
2121

2222

23-
.. warning::
23+
.. note::
2424

25-
The :mod:`xml.etree.ElementTree` module is not secure against
26-
maliciously constructed data. If you need to parse untrusted or
27-
unauthenticated data see :ref:`xml-vulnerabilities`.
25+
If you need to parse untrusted or unauthenticated data, see
26+
:ref:`xml-security`.
2827

2928
Tutorial
3029
--------

_sources/library/xml.rst.txt

Lines changed: 12 additions & 64 deletions
Original file line numberDiff line numberDiff line change
@@ -15,12 +15,10 @@ XML Processing Modules
1515

1616
Python's interfaces for processing XML are grouped in the ``xml`` package.
1717

18-
.. warning::
18+
.. note::
1919

20-
The XML modules are not secure against erroneous or maliciously
21-
constructed data. If you need to parse untrusted or
22-
unauthenticated data see the :ref:`xml-vulnerabilities` and
23-
:ref:`defusedxml-package` sections.
20+
If you need to parse untrusted or unauthenticated data, see
21+
:ref:`xml-security`.
2422

2523
It is important to note that modules in the :mod:`xml` package require that
2624
there be at least one SAX-compliant XML parser available. The Expat parser is
@@ -47,46 +45,22 @@ The XML handling submodules are:
4745
* :mod:`xml.parsers.expat`: the Expat parser binding
4846

4947

48+
.. _xml-security:
5049
.. _xml-vulnerabilities:
5150

52-
XML vulnerabilities
53-
-------------------
51+
XML security
52+
------------
5453

55-
The XML processing modules are not secure against maliciously constructed data.
5654
An attacker can abuse XML features to carry out denial of service attacks,
5755
access local files, generate network connections to other machines, or
5856
circumvent firewalls.
5957

60-
The following table gives an overview of the known attacks and whether
61-
the various modules are vulnerable to them.
62-
63-
========================= ================== ================== ================== ================== ==================
64-
kind sax etree minidom pulldom xmlrpc
65-
========================= ================== ================== ================== ================== ==================
66-
billion laughs **Vulnerable** (1) **Vulnerable** (1) **Vulnerable** (1) **Vulnerable** (1) **Vulnerable** (1)
67-
quadratic blowup **Vulnerable** (1) **Vulnerable** (1) **Vulnerable** (1) **Vulnerable** (1) **Vulnerable** (1)
68-
external entity expansion Safe (5) Safe (2) Safe (3) Safe (5) Safe (4)
69-
`DTD`_ retrieval Safe (5) Safe Safe Safe (5) Safe
70-
decompression bomb Safe Safe Safe Safe **Vulnerable**
71-
large tokens **Vulnerable** (6) **Vulnerable** (6) **Vulnerable** (6) **Vulnerable** (6) **Vulnerable** (6)
72-
========================= ================== ================== ================== ================== ==================
73-
74-
1. Expat 2.4.1 and newer is not vulnerable to the "billion laughs" and
75-
"quadratic blowup" vulnerabilities. Items still listed as vulnerable due to
76-
potential reliance on system-provided libraries. Check
77-
:const:`!pyexpat.EXPAT_VERSION`.
78-
2. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a
79-
:exc:`~xml.etree.ElementTree.ParseError` when an entity occurs.
80-
3. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns
81-
the unexpanded entity verbatim.
82-
4. :mod:`xmlrpc.client` doesn't expand external entities and omits them.
83-
5. Since Python 3.7.1, external general entities are no longer processed by
84-
default.
85-
6. Expat 2.6.0 and newer is not vulnerable to denial of service
86-
through quadratic runtime caused by parsing large tokens.
87-
Items still listed as vulnerable due to
88-
potential reliance on system-provided libraries. Check
89-
:const:`!pyexpat.EXPAT_VERSION`.
58+
Expat versions lower that 2.6.0 may be vulnerable to "billion laughs",
59+
"quadratic blowup" and "large tokens". Python may be vulnerable if it uses such
60+
older versions of Expat as a system-provided library.
61+
Check :const:`!pyexpat.EXPAT_VERSION`.
62+
63+
:mod:`xmlrpc` is **vulnerable** to the "decompression bomb" attack.
9064

9165

9266
billion laughs / exponential entity expansion
@@ -103,16 +77,6 @@ quadratic blowup entity expansion
10377
efficient as the exponential case but it avoids triggering parser countermeasures
10478
that forbid deeply nested entities.
10579

106-
external entity expansion
107-
Entity declarations can contain more than just text for replacement. They can
108-
also point to external resources or local files. The XML
109-
parser accesses the resource and embeds the content into the XML document.
110-
111-
`DTD`_ retrieval
112-
Some XML libraries like Python's :mod:`xml.dom.pulldom` retrieve document type
113-
definitions from remote or local locations. The feature has similar
114-
implications as the external entity expansion issue.
115-
11680
decompression bomb
11781
Decompression bombs (aka `ZIP bomb`_) apply to all XML libraries
11882
that can parse compressed XML streams such as gzipped HTTP streams or
@@ -126,21 +90,5 @@ large tokens
12690
be used to cause denial of service in the application parsing XML.
12791
The issue is known as :cve:`2023-52425`.
12892

129-
The documentation for :pypi:`defusedxml` on PyPI has further information about
130-
all known attack vectors with examples and references.
131-
132-
.. _defusedxml-package:
133-
134-
The :mod:`!defusedxml` Package
135-
------------------------------
136-
137-
:pypi:`defusedxml` is a pure Python package with modified subclasses of all stdlib
138-
XML parsers that prevent any potentially malicious operation. Use of this
139-
package is recommended for any server code that parses untrusted XML data. The
140-
package also ships with example exploits and extended documentation on more
141-
XML exploits such as XPath injection.
142-
143-
14493
.. _Billion Laughs: https://en.wikipedia.org/wiki/Billion_laughs
14594
.. _ZIP bomb: https://en.wikipedia.org/wiki/Zip_bomb
146-
.. _DTD: https://en.wikipedia.org/wiki/Document_type_definition

_sources/library/xml.sax.rst.txt

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,10 @@ SAX exceptions and the convenience functions which will be most used by users of
1818
the SAX API.
1919

2020

21-
.. warning::
21+
.. note::
2222

23-
The :mod:`xml.sax` module is not secure against maliciously
24-
constructed data. If you need to parse untrusted or unauthenticated data see
25-
:ref:`xml-vulnerabilities`.
23+
If you need to parse untrusted or unauthenticated data, see
24+
:ref:`xml-security`.
2625

2726
.. versionchanged:: 3.7.1
2827

_sources/library/xmlrpc.client.rst.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,8 @@ between conformable Python objects and XML on the wire.
2424
.. warning::
2525

2626
The :mod:`xmlrpc.client` module is not secure against maliciously
27-
constructed data. If you need to parse untrusted or unauthenticated data see
28-
:ref:`xml-vulnerabilities`.
27+
constructed data. If you need to parse untrusted or unauthenticated data,
28+
see :ref:`xml-security`.
2929

3030
.. versionchanged:: 3.5
3131

0 commit comments

Comments
 (0)