Skip to content

gh-128555: Add 'context' keyword parameter to Thread. #128209

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions Doc/library/decimal.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1884,13 +1884,20 @@ the current thread.

If :func:`setcontext` has not been called before :func:`getcontext`, then
:func:`getcontext` will automatically create a new context for use in the
current thread.

The new context is copied from a prototype context called *DefaultContext*. To
control the defaults so that each thread will use the same values throughout the
application, directly modify the *DefaultContext* object. This should be done
*before* any threads are started so that there won't be a race condition between
threads calling :func:`getcontext`. For example::
current thread. New context objects have default values set from the
:data:`decimal.DefaultContext` object.

The :data:`sys.flags.thread_inherit_context` flag affects the context for
new threads. If the flag is false, new threads will start with an empty
context. In this case, :func:`getcontext` will create a new context object
when called and use the default values from *DefaultContext*. If the flag
is true, new threads will start with a copy of context from the caller of
:meth:`Thread.start`.

To control the defaults so that each thread will use the same values throughout
the application, directly modify the *DefaultContext* object. This should be
done *before* any threads are started so that there won't be a race condition
between threads calling :func:`getcontext`. For example::

# Set applicationwide defaults for all threads about to be launched
DefaultContext.prec = 12
Expand Down
16 changes: 15 additions & 1 deletion Doc/library/sys.rst
Original file line number Diff line number Diff line change
Expand Up @@ -535,7 +535,8 @@ always available. Unless explicitly noted otherwise, all variables are read-only
.. data:: flags

The :term:`named tuple` *flags* exposes the status of command line
flags. The attributes are read only.
flags. Flags should only be accessed only by name and not by index. The
attributes are read only.

.. list-table::

Expand Down Expand Up @@ -594,6 +595,13 @@ always available. Unless explicitly noted otherwise, all variables are read-only
* - .. attribute:: flags.warn_default_encoding
- :option:`-X warn_default_encoding <-X>`

* - .. attribute:: flags.gil
- :option:`-X gil <-X>` and :envvar:`PYTHON_GIL`

* - .. attribute:: flags.thread_inherit_context
- :option:`-X thread_inherit_context <-X>` and
:envvar:`PYTHON_THREAD_INHERIT_CONTEXT`

.. versionchanged:: 3.2
Added ``quiet`` attribute for the new :option:`-q` flag.

Expand All @@ -620,6 +628,12 @@ always available. Unless explicitly noted otherwise, all variables are read-only
.. versionchanged:: 3.11
Added the ``int_max_str_digits`` attribute.

.. versionchanged:: 3.13
Added the ``gil`` attribute.

.. versionchanged:: 3.14
Added the ``thread_inherit_context`` attribute.


.. data:: float_info

Expand Down
15 changes: 14 additions & 1 deletion Doc/library/threading.rst
Original file line number Diff line number Diff line change
Expand Up @@ -334,7 +334,7 @@ since it is impossible to detect the termination of alien threads.


.. class:: Thread(group=None, target=None, name=None, args=(), kwargs={}, *, \
daemon=None)
daemon=None, context=None)

This constructor should always be called with keyword arguments. Arguments
are:
Expand All @@ -359,6 +359,16 @@ since it is impossible to detect the termination of alien threads.
If ``None`` (the default), the daemonic property is inherited from the
current thread.

*context* is the :class:`~contextvars.Context` value to use when starting
the thread. The default value is ``None`` which indicates that the
:data:`sys.flags.thread_inherit_context` flag controls the behaviour. If
the flag is true, threads will start with a copy of the context of the
caller of :meth:`~Thread.start`. If false, they will start with an empty
context. To explicitly start with an empty context, pass a new instance of
:class:`~contextvars.Context()`. To explicitly start with a copy of the
current context, pass the value from :func:`~contextvars.copy_context`. The
flag defaults true on free-threaded builds and false otherwise.

If the subclass overrides the constructor, it must make sure to invoke the
base class constructor (``Thread.__init__()``) before doing anything else to
the thread.
Expand All @@ -369,6 +379,9 @@ since it is impossible to detect the termination of alien threads.
.. versionchanged:: 3.10
Use the *target* name if *name* argument is omitted.

.. versionchanged:: 3.14
Added the *context* parameter.

.. method:: start()

Start the thread's activity.
Expand Down
19 changes: 19 additions & 0 deletions Doc/using/cmdline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -628,6 +628,15 @@ Miscellaneous options

.. versionadded:: 3.13

* :samp:`-X thread_inherit_context={0,1}` causes :class:`~threading.Thread`
to, by default, use a copy of context of of the caller of
``Thread.start()`` when starting. Otherwise, threads will start
with an empty context. If unset, the value of this option defaults
to ``1`` on free-threaded builds and to ``0`` otherwise. See also
:envvar:`PYTHON_THREAD_INHERIT_CONTEXT`.

.. versionadded:: 3.14

It also allows passing arbitrary values and retrieving them through the
:data:`sys._xoptions` dictionary.

Expand Down Expand Up @@ -1221,6 +1230,16 @@ conflict.

.. versionadded:: 3.13

.. envvar:: PYTHON_THREAD_INHERIT_CONTEXT

If this variable is set to ``1`` then :class:`~threading.Thread` will,
by default, use a copy of context of of the caller of ``Thread.start()``
when starting. Otherwise, new threads will start with an empty context.
If unset, this variable defaults to ``1`` on free-threaded builds and to
``0`` otherwise. See also :option:`-X thread_inherit_context<-X>`.

.. versionadded:: 3.14

Debug-mode variables
~~~~~~~~~~~~~~~~~~~~

Expand Down
1 change: 1 addition & 0 deletions Include/cpython/initconfig.h
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,7 @@ typedef struct PyConfig {
int use_frozen_modules;
int safe_path;
int int_max_str_digits;
int thread_inherit_context;
#ifdef __APPLE__
int use_system_logger;
#endif
Expand Down
10 changes: 7 additions & 3 deletions Lib/test/test_capi/test_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ def test_config_get(self):
("filesystem_errors", str, None),
("hash_seed", int, None),
("home", str | None, None),
("thread_inherit_context", int, None),
("import_time", bool, None),
("inspect", bool, None),
("install_signal_handlers", bool, None),
Expand Down Expand Up @@ -98,7 +99,7 @@ def test_config_get(self):
]
if support.Py_DEBUG:
options.append(("run_presite", str | None, None))
if sysconfig.get_config_var('Py_GIL_DISABLED'):
if support.Py_GIL_DISABLED:
options.append(("enable_gil", int, None))
options.append(("tlbc_enabled", int, None))
if support.MS_WINDOWS:
Expand Down Expand Up @@ -170,7 +171,7 @@ def test_config_get_sys_flags(self):
("warn_default_encoding", "warn_default_encoding", False),
("safe_path", "safe_path", False),
("int_max_str_digits", "int_max_str_digits", False),
# "gil" is tested below
# "gil" and "thread_inherit_context" are tested below
):
with self.subTest(flag=flag, name=name, negate=negate):
value = config_get(name)
Expand All @@ -182,11 +183,14 @@ def test_config_get_sys_flags(self):
config_get('use_hash_seed') == 0
or config_get('hash_seed') != 0)

if sysconfig.get_config_var('Py_GIL_DISABLED'):
if support.Py_GIL_DISABLED:
value = config_get('enable_gil')
expected = (value if value != -1 else None)
self.assertEqual(sys.flags.gil, expected)

expected_inherit_context = 1 if support.Py_GIL_DISABLED else 0
self.assertEqual(sys.flags.thread_inherit_context, expected_inherit_context)

def test_config_get_non_existent(self):
# Test PyConfig_Get() on non-existent option name
config_get = _testcapi.config_get
Expand Down
55 changes: 55 additions & 0 deletions Lib/test/test_context.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import sys
import collections.abc
import concurrent.futures
import contextvars
Expand Down Expand Up @@ -383,6 +384,60 @@ def sub(num):
tp.shutdown()
self.assertEqual(results, list(range(10)))

@isolated_context
@threading_helper.requires_working_threading()
def test_context_thread_inherit(self):
import threading

cvar = contextvars.ContextVar('cvar')

def run_context_none():
if sys.flags.thread_inherit_context:
expected = 1
else:
expected = None
self.assertEqual(cvar.get(None), expected)

# By default, context is inherited based on the
# sys.flags.thread_inherit_context option.
cvar.set(1)
thread = threading.Thread(target=run_context_none)
thread.start()
thread.join()

# Passing 'None' explicitly should have same behaviour as not
# passing parameter.
thread = threading.Thread(target=run_context_none, context=None)
thread.start()
thread.join()

# An explicit Context value can also be passed
custom_ctx = contextvars.Context()
custom_var = None

def setup_context():
nonlocal custom_var
custom_var = contextvars.ContextVar('custom')
custom_var.set(2)

custom_ctx.run(setup_context)

def run_custom():
self.assertEqual(custom_var.get(), 2)

thread = threading.Thread(target=run_custom, context=custom_ctx)
thread.start()
thread.join()

# You can also pass a new Context() object to start with an empty context
def run_empty():
with self.assertRaises(LookupError):
cvar.get()

thread = threading.Thread(target=run_empty, context=contextvars.Context())
thread.start()
thread.join()


# HAMT Tests

Expand Down
10 changes: 8 additions & 2 deletions Lib/test/test_decimal.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
import random
import inspect
import threading
import contextvars


if sys.platform == 'darwin':
Expand Down Expand Up @@ -1725,8 +1726,13 @@ def test_threading(self):
self.finish1 = threading.Event()
self.finish2 = threading.Event()

th1 = threading.Thread(target=thfunc1, args=(self,))
th2 = threading.Thread(target=thfunc2, args=(self,))
# This test wants to start threads with an empty context, no matter
# the setting of sys.flags.thread_inherit_context. We pass the
# 'context' argument explicitly with an empty context instance.
th1 = threading.Thread(target=thfunc1, args=(self,),
context=contextvars.Context())
th2 = threading.Thread(target=thfunc2, args=(self,),
context=contextvars.Context())

th1.start()
th2.start()
Expand Down
7 changes: 6 additions & 1 deletion Lib/test/test_embed.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
INIT_LOOPS = 4
MAX_HASH_SEED = 4294967295

ABI_THREAD = 't' if sysconfig.get_config_var('Py_GIL_DISABLED') else ''
ABI_THREAD = 't' if support.Py_GIL_DISABLED else ''
# PLATSTDLIB_LANDMARK copied from Modules/getpath.py
if os.name == 'nt':
PLATSTDLIB_LANDMARK = f'{sys.platlibdir}'
Expand All @@ -60,6 +60,10 @@
PLATSTDLIB_LANDMARK = (f'{sys.platlibdir}/python{VERSION_MAJOR}.'
f'{VERSION_MINOR}{ABI_THREAD}/lib-dynload')

if support.Py_GIL_DISABLED:
DEFAULT_THREAD_INHERIT_CONTEXT = 1
else:
DEFAULT_THREAD_INHERIT_CONTEXT = 0

# If we are running from a build dir, but the stdlib has been installed,
# some tests need to expect different results.
Expand Down Expand Up @@ -586,6 +590,7 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase):
'tracemalloc': 0,
'perf_profiling': 0,
'import_time': False,
'thread_inherit_context': DEFAULT_THREAD_INHERIT_CONTEXT,
'code_debug_ranges': True,
'show_ref_count': False,
'dump_refs': False,
Expand Down
5 changes: 3 additions & 2 deletions Lib/test/test_sys.py
Original file line number Diff line number Diff line change
Expand Up @@ -1845,8 +1845,9 @@ def test_pythontypes(self):
# symtable entry
# XXX
# sys.flags
# FIXME: The +1 will not be necessary once gh-122575 is fixed
check(sys.flags, vsize('') + self.P * (1 + len(sys.flags)))
# FIXME: The +2 is for the 'gil' and 'thread_inherit_context' flags and
# will not be necessary once gh-122575 is fixed
check(sys.flags, vsize('') + self.P * (2 + len(sys.flags)))

def test_asyncgen_hooks(self):
old = sys.get_asyncgen_hooks()
Expand Down
24 changes: 22 additions & 2 deletions Lib/threading.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import os as _os
import sys as _sys
import _thread
import _contextvars

from time import monotonic as _time
from _weakrefset import WeakSet
Expand Down Expand Up @@ -871,7 +872,7 @@ class Thread:
_initialized = False

def __init__(self, group=None, target=None, name=None,
args=(), kwargs=None, *, daemon=None):
args=(), kwargs=None, *, daemon=None, context=None):
"""This constructor should always be called with keyword arguments. Arguments are:

*group* should be None; reserved for future extension when a ThreadGroup
Expand All @@ -888,6 +889,14 @@ class is implemented.
*kwargs* is a dictionary of keyword arguments for the target
invocation. Defaults to {}.

*context* is the contextvars.Context value to use for the thread.
The default value is None, which means to check
sys.flags.thread_inherit_context. If that flag is true, use a copy
of the context of the caller. If false, use an empty context. To
explicitly start with an empty context, pass a new instance of
contextvars.Context(). To explicitly start with a copy of the current
context, pass the value from contextvars.copy_context().

If a subclass overrides the constructor, it must make sure to invoke
the base class constructor (Thread.__init__()) before doing anything
else to the thread.
Expand Down Expand Up @@ -917,6 +926,7 @@ class is implemented.
self._daemonic = daemon
else:
self._daemonic = current_thread().daemon
self._context = context
self._ident = None
if _HAVE_THREAD_NATIVE_ID:
self._native_id = None
Expand Down Expand Up @@ -972,6 +982,16 @@ def start(self):

with _active_limbo_lock:
_limbo[self] = self

if self._context is None:
# No context provided
if _sys.flags.thread_inherit_context:
# start with a copy of the context of the caller
self._context = _contextvars.copy_context()
else:
# start with an empty context
self._context = _contextvars.Context()

try:
# Start joinable thread
_start_joinable_thread(self._bootstrap, handle=self._handle,
Expand Down Expand Up @@ -1051,7 +1071,7 @@ def _bootstrap_inner(self):
_sys.setprofile(_profile_hook)

try:
self.run()
self._context.run(self.run)
except:
self._invoke_excepthook(self)
finally:
Expand Down
1 change: 1 addition & 0 deletions Makefile.pre.in
Original file line number Diff line number Diff line change
Expand Up @@ -420,6 +420,7 @@ PARSER_HEADERS= \
# Python

PYTHON_OBJS= \
Python/_contextvars.o \
Python/_warnings.o \
Python/Python-ast.o \
Python/Python-tokenize.o \
Expand Down
Loading