Skip to content

Commit ce3505b

Browse files
Add a whatsnew entry.
1 parent 88e47fb commit ce3505b

File tree

1 file changed

+95
-0
lines changed

1 file changed

+95
-0
lines changed

Doc/whatsnew/3.14.rst

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,7 @@ and improvements in user-friendliness and correctness.
8383
.. PEP-sized items next.
8484
8585
* :ref:`PEP 649 and 749: deferred evaluation of annotations <whatsnew314-pep649>`
86+
* :ref:`PEP 734: Multiple Interpreters in the Stdlib <whatsnew314-pep734>`
8687
* :ref:`PEP 741: Python Configuration C API <whatsnew314-pep741>`
8788
* :ref:`PEP 750: Template strings <whatsnew314-pep750>`
8889
* :ref:`PEP 758: Allow except and except* expressions without parentheses <whatsnew314-pep758>`
@@ -123,6 +124,98 @@ of Python. See :ref:`below <whatsnew314-refcount>` for details.
123124
New features
124125
============
125126

127+
.. _whatsnew314-pep734:
128+
129+
PEP 734: Multiple Interpreters in the Stdlib
130+
--------------------------------------------
131+
132+
The CPython runtime supports running multiple copies of Python in the
133+
same process simultaneously and has done so for over 20 years.
134+
Each of these separate copies is called an "interpreter".
135+
However, the feature has been available only through the C-API.
136+
137+
That limitation is removed in the 3.14 release,
138+
with the new :mod:`interpreters` module!
139+
140+
There are at least two notable reasons why using multiple interpreters
141+
is worth considering:
142+
143+
* they support a new (to Python), human-friendly concurrency model
144+
* true multi-core parallelism
145+
146+
For some use cases, concurrency in software enables efficiency and
147+
can simplify software, at a high level. At the same time, implementing
148+
and maintaining all but the simplest concurrency is often a struggle
149+
for the human brain. That especially applies to plain threads
150+
(e.g. :mod:`threading`), where all memory is shared between all threads.
151+
152+
With multiple isolated interpreters, you can take advantage of a class
153+
of concurrency models, like CSP or the actor model, that have found
154+
success in other programming languages, like Smalltalk, Erlang,
155+
Haskell, and Go. Think of multiple interpreters like threads
156+
but with opt-in sharing.
157+
158+
Regarding multi-core parallelism: as of the 3.12 release, interpreters
159+
are now sufficiently isolated from one another to be used in parallel.
160+
(See :pep:`684`.) This unlocks a variety of CPU-intensive use cases
161+
for Python that were limited by the :term:`GIL`.
162+
163+
Using multiple interpreters is similar in many ways to
164+
:mod:`multiprocessing`, in that they both provide isolated logical
165+
"processes" that can run in parallel, with no sharing by default.
166+
. However, when using multiple interpreters, an application will use
167+
fewer system resources and will operate more efficiently (since it
168+
stays within the same process). Think of multiple interpreters as
169+
having the isolation of processes with the efficiency of threads.
170+
171+
.. XXX Add an example or two.
172+
.. XXX Link to the not-yet-added HOWTO doc.
173+
174+
While the feature has been around for decades, multiple interpreters
175+
have not been used widely, due to low awareness and the lack of a stdlib
176+
module. Consequently, they currently have several notable limitations,
177+
which will improve significantly now that the feature is finally
178+
going mainstream.
179+
180+
Current limitations:
181+
182+
* starting each interpreter has not been optimized yet
183+
* each interpreter uses more memory than necessary
184+
(we will be working next on extensive internal sharing between
185+
interpreters)
186+
* there aren't many options *yet* for truly sharing objects or other
187+
data between interpreters (other than :type:`memoryview`)
188+
* many extension modules on PyPI are not compatible with multiple
189+
interpreters yet (stdlib extension modules *are* compatible)
190+
* the approach to writing applications that use multiple isolated
191+
interpreters is mostly unfamiliar to Python users, for now
192+
193+
The impact of these limitations will depend on future CPython
194+
improvements, how interpreters are used, and what the community solves
195+
through PyPI packages. Depending on the use case, the limitations may
196+
not have much impact, so try it out!
197+
198+
Furthermore, future CPython releases will reduce or eliminate overhead
199+
and provide utilities that are less appropriate on PyPI. In the
200+
meantime, most of the limitations can also be addressed through
201+
extension modules, meaning PyPI packages can fill any gap for 3.14, and
202+
even back to 3.12 where interpreters were finally properly isolated and
203+
stopped sharing the :term:`GIL`. Likewise, we expect to slowly see
204+
libraries on PyPI for high-level abstractions on top of interpreters.
205+
206+
Regarding extension modules, work is in progress to update some PyPI
207+
projects, as well as tools like Cython, PyBind11, Nanobind, and Py03.
208+
The steps for isolating an extension module are found at
209+
:ref:`isolating-extensions-howto`. Isolating a module has a lot of
210+
overlap with what is required to support
211+
:ref:`free-threadeding <whatsnew314-free-threaded-cpython>`,
212+
so the ongoing work in the community in that area will help accelerate
213+
support for multiple interpreters.
214+
215+
Also added in 3.14: :ref:`concurrent.futures.InterpreterPoolExecutor
216+
<whatsnew314-concurrent-futures-interp-pool>`.
217+
218+
126219
.. _whatsnew314-pep750:
127220

128221
PEP 750: Template strings
@@ -1108,6 +1201,8 @@ calendar
11081201
concurrent.futures
11091202
------------------
11101203

1204+
.. _whatsnew314-concurrent-futures-interp-pool:
1205+
11111206
* Add :class:`~concurrent.futures.InterpreterPoolExecutor`,
11121207
which exposes "subinterpreters" (multiple Python interpreters in the
11131208
same process) to Python code. This is separate from the proposed API

0 commit comments

Comments
 (0)