@@ -83,6 +83,7 @@ and improvements in user-friendliness and correctness.
83
83
.. PEP-sized items next.
84
84
85
85
* :ref: `PEP 649 and 749: deferred evaluation of annotations <whatsnew314-pep649 >`
86
+ * :ref: `PEP 734: Multiple Interpreters in the Stdlib <whatsnew314-pep734 >`
86
87
* :ref: `PEP 741: Python Configuration C API <whatsnew314-pep741 >`
87
88
* :ref: `PEP 750: Template strings <whatsnew314-pep750 >`
88
89
* :ref: `PEP 758: Allow except and except* expressions without parentheses <whatsnew314-pep758 >`
@@ -123,6 +124,98 @@ of Python. See :ref:`below <whatsnew314-refcount>` for details.
123
124
New features
124
125
============
125
126
127
+ .. _whatsnew314-pep734 :
128
+
129
+ PEP 734: Multiple Interpreters in the Stdlib
130
+ --------------------------------------------
131
+
132
+ The CPython runtime supports running multiple copies of Python in the
133
+ same process simultaneously and has done so for over 20 years.
134
+ Each of these separate copies is called an "interpreter".
135
+ However, the feature has been available only through the C-API.
136
+
137
+ That limitation is removed in the 3.14 release,
138
+ with the new :mod: `interpreters ` module!
139
+
140
+ There are at least two notable reasons why using multiple interpreters
141
+ is worth considering:
142
+
143
+ * they support a new (to Python), human-friendly concurrency model
144
+ * true multi-core parallelism
145
+
146
+ For some use cases, concurrency in software enables efficiency and
147
+ can simplify software, at a high level. At the same time, implementing
148
+ and maintaining all but the simplest concurrency is often a struggle
149
+ for the human brain. That especially applies to plain threads
150
+ (e.g. :mod: `threading `), where all memory is shared between all threads.
151
+
152
+ With multiple isolated interpreters, you can take advantage of a class
153
+ of concurrency models, like CSP or the actor model, that have found
154
+ success in other programming languages, like Smalltalk, Erlang,
155
+ Haskell, and Go. Think of multiple interpreters like threads
156
+ but with opt-in sharing.
157
+
158
+ Regarding multi-core parallelism: as of the 3.12 release, interpreters
159
+ are now sufficiently isolated from one another to be used in parallel.
160
+ (See :pep: `684 `.) This unlocks a variety of CPU-intensive use cases
161
+ for Python that were limited by the :term: `GIL `.
162
+
163
+ Using multiple interpreters is similar in many ways to
164
+ :mod: `multiprocessing `, in that they both provide isolated logical
165
+ "processes" that can run in parallel, with no sharing by default.
166
+ . However, when using multiple interpreters, an application will use
167
+ fewer system resources and will operate more efficiently (since it
168
+ stays within the same process). Think of multiple interpreters as
169
+ having the isolation of processes with the efficiency of threads.
170
+
171
+ .. XXX Add an example or two.
172
+ .. XXX Link to the not-yet-added HOWTO doc.
173
+
174
+ While the feature has been around for decades, multiple interpreters
175
+ have not been used widely, due to low awareness and the lack of a stdlib
176
+ module. Consequently, they currently have several notable limitations,
177
+ which will improve significantly now that the feature is finally
178
+ going mainstream.
179
+
180
+ Current limitations:
181
+
182
+ * starting each interpreter has not been optimized yet
183
+ * each interpreter uses more memory than necessary
184
+ (we will be working next on extensive internal sharing between
185
+ interpreters)
186
+ * there aren't many options *yet * for truly sharing objects or other
187
+ data between interpreters (other than :type: `memoryview `)
188
+ * many extension modules on PyPI are not compatible with multiple
189
+ interpreters yet (stdlib extension modules *are * compatible)
190
+ * the approach to writing applications that use multiple isolated
191
+ interpreters is mostly unfamiliar to Python users, for now
192
+
193
+ The impact of these limitations will depend on future CPython
194
+ improvements, how interpreters are used, and what the community solves
195
+ through PyPI packages. Depending on the use case, the limitations may
196
+ not have much impact, so try it out!
197
+
198
+ Furthermore, future CPython releases will reduce or eliminate overhead
199
+ and provide utilities that are less appropriate on PyPI. In the
200
+ meantime, most of the limitations can also be addressed through
201
+ extension modules, meaning PyPI packages can fill any gap for 3.14, and
202
+ even back to 3.12 where interpreters were finally properly isolated and
203
+ stopped sharing the :term: `GIL `. Likewise, we expect to slowly see
204
+ libraries on PyPI for high-level abstractions on top of interpreters.
205
+
206
+ Regarding extension modules, work is in progress to update some PyPI
207
+ projects, as well as tools like Cython, PyBind11, Nanobind, and Py03.
208
+ The steps for isolating an extension module are found at
209
+ :ref: `isolating-extensions-howto `. Isolating a module has a lot of
210
+ overlap with what is required to support
211
+ :ref: `free-threadeding <whatsnew314-free-threaded-cpython >`,
212
+ so the ongoing work in the community in that area will help accelerate
213
+ support for multiple interpreters.
214
+
215
+ Also added in 3.14: :ref: `concurrent.futures.InterpreterPoolExecutor
216
+ <whatsnew314-concurrent-futures-interp-pool>`.
217
+
218
+
126
219
.. _whatsnew314-pep750 :
127
220
128
221
PEP 750: Template strings
@@ -1108,6 +1201,8 @@ calendar
1108
1201
concurrent.futures
1109
1202
------------------
1110
1203
1204
+ .. _whatsnew314-concurrent-futures-interp-pool :
1205
+
1111
1206
* Add :class: `~concurrent.futures.InterpreterPoolExecutor `,
1112
1207
which exposes "subinterpreters" (multiple Python interpreters in the
1113
1208
same process) to Python code. This is separate from the proposed API
0 commit comments