Skip to content

gh-137173: Allow signal handling in isolated subinterpreters #137174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open
9 changes: 9 additions & 0 deletions Doc/c-api/init.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1580,6 +1580,15 @@ function. You can create and destroy them using the following functions:
If this is :c:macro:`PyInterpreterConfig_OWN_GIL` then
:c:member:`PyInterpreterConfig.use_main_obmalloc` must be ``0``.

.. c:member:: int can_handle_signals

If this is ``0``, then the interpreter will ignore incoming signals when
it is running on the main thread. These signals will instead be handled
by the next interpreter in the main thread that is capable of handling
signals.

.. versionadded:: next


.. c:function:: PyStatus Py_NewInterpreterFromConfig(PyThreadState **tstate_p, const PyInterpreterConfig *config)

Expand Down
11 changes: 6 additions & 5 deletions Doc/library/signal.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,12 +61,13 @@ This has consequences:
Signals and threads
^^^^^^^^^^^^^^^^^^^

Python signal handlers are always executed in the main Python thread of the main interpreter,
Python signal handlers are always executed in the main Python thread of
intepreters that support signal handling (:c:member:`PyInterpreterConfig.can_handle_signals`),
even if the signal was received in another thread. This means that signals
can't be used as a means of inter-thread communication. You can use
the synchronization primitives from the :mod:`threading` module instead.

Besides, only the main thread of the main interpreter is allowed to set a new signal handler.
Besides, only the main thread is allowed to set a new signal handler.


Module contents
Expand Down Expand Up @@ -421,7 +422,7 @@ The :mod:`signal` module defines the following functions:
same process as the caller. The target thread can be executing any code
(Python or not). However, if the target thread is executing the Python
interpreter, the Python signal handlers will be :ref:`executed by the main
thread of the main interpreter <signals-and-threads>`. Therefore, the only point of sending a
thread of a supporting interpreter <signals-and-threads>`. Therefore, the only point of sending a
signal to a particular Python thread would be to force a running system call
to fail with :exc:`InterruptedError`.

Expand Down Expand Up @@ -523,7 +524,7 @@ The :mod:`signal` module defines the following functions:
any bytes from *fd* before calling poll or select again.

When threads are enabled, this function can only be called
from :ref:`the main thread of the main interpreter <signals-and-threads>`;
from :ref:`the main thread of a supporting interpreter <signals-and-threads>`;
attempting to call it from other threads will cause a :exc:`ValueError`
exception to be raised.

Expand Down Expand Up @@ -578,7 +579,7 @@ The :mod:`signal` module defines the following functions:
above). (See the Unix man page :manpage:`signal(2)` for further information.)

When threads are enabled, this function can only be called
from :ref:`the main thread of the main interpreter <signals-and-threads>`;
from :ref:`the main thread of a supporting interpreter <signals-and-threads>`;
attempting to call it from other threads will cause a :exc:`ValueError`
exception to be raised.

Expand Down
5 changes: 5 additions & 0 deletions Doc/whatsnew/3.15.rst
Original file line number Diff line number Diff line change
Expand Up @@ -487,6 +487,11 @@ New features
a string. See the documentation for caveats.
(Contributed by Petr Viktorin in :gh:`131510`)

* Subinterpreters now have the option to handle signals through the
:c:member:`PyInterpreterConfig.can_handle_signals` setting. Only
subinterpreters running in the main thread will be able to handle
signals.


Porting to Python 3.15
----------------------
Expand Down
3 changes: 3 additions & 0 deletions Include/cpython/pylifecycle.h
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ typedef struct {
int allow_daemon_threads;
int check_multi_interp_extensions;
int gil;
int can_handle_signals;
} PyInterpreterConfig;

#define _PyInterpreterConfig_INIT \
Expand All @@ -58,6 +59,7 @@ typedef struct {
.allow_daemon_threads = 0, \
.check_multi_interp_extensions = 1, \
.gil = PyInterpreterConfig_OWN_GIL, \
.can_handle_signals = 1, \
}

// gh-117649: The free-threaded build does not currently support single-phase
Expand All @@ -78,6 +80,7 @@ typedef struct {
.allow_daemon_threads = 1, \
.check_multi_interp_extensions = _PyInterpreterConfig_LEGACY_CHECK_MULTI_INTERP_EXTENSIONS, \
.gil = PyInterpreterConfig_SHARED_GIL, \
.can_handle_signals = 0, \
}

PyAPI_FUNC(PyStatus) Py_NewInterpreterFromConfig(
Expand Down
3 changes: 3 additions & 0 deletions Include/internal/pycore_interp.h
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,9 @@ might not be allowed in the current interpreter (i.e. os.fork() would fail).
/* Set if os.exec*() is allowed. */
#define Py_RTFLAGS_EXEC (1UL << 16)

/* Set if signal handling is allowed. */
#define Py_RTFLAGS_CAN_HANDLE_SIGNALS (1UL << 17)

extern int _PyInterpreterState_HasFeature(PyInterpreterState *interp,
unsigned long feature);

Expand Down
6 changes: 4 additions & 2 deletions Include/internal/pycore_pystate.h
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ extern "C" {
#include "pycore_pythonrun.h" // _PyOS_STACK_MARGIN_SHIFT
#include "pycore_typedefs.h" // _PyRuntimeState
#include "pycore_tstate.h"
#include "pycore_interp.h" // _PyInterpreterState_HasFeature()


// Values for PyThreadState.state. A thread must be in the "attached" state
Expand Down Expand Up @@ -78,11 +79,12 @@ extern void _PyInterpreterState_ReinitRunningMain(PyThreadState *);
extern const PyConfig* _Py_GetMainConfig(void);


/* Only handle signals on the main thread of the main interpreter. */
/* Only handle signals on the main thread of an interpreter that supports it. */
static inline int
_Py_ThreadCanHandleSignals(PyInterpreterState *interp)
{
return (_Py_IsMainThread() && _Py_IsMainInterpreter(interp));
return (_Py_IsMainThread()
&& _PyInterpreterState_HasFeature(interp, Py_RTFLAGS_CAN_HANDLE_SIGNALS));
}


Expand Down
1 change: 1 addition & 0 deletions Lib/test/test_capi/test_getargs.py
Original file line number Diff line number Diff line change
Expand Up @@ -1445,6 +1445,7 @@ def test_gh_119213(self):
use_main_obmalloc=False,
gil=2,
check_multi_interp_extensions=True,
can_handle_signals=True,
)
rc = support.run_in_subinterp_with_config(script, **config)
assert rc == 0
Expand Down
47 changes: 29 additions & 18 deletions Lib/test/test_capi/test_misc.py
Original file line number Diff line number Diff line change
Expand Up @@ -1730,8 +1730,9 @@ def test_configured_settings(self):
DAEMON_THREADS = 1<<11
FORK = 1<<15
EXEC = 1<<16
SIGNALS = 1<<17
ALL_FLAGS = (OBMALLOC | FORK | EXEC | THREADS | DAEMON_THREADS
| EXTENSIONS);
| EXTENSIONS | SIGNALS);

features = [
'obmalloc',
Expand All @@ -1741,23 +1742,25 @@ def test_configured_settings(self):
'daemon_threads',
'extensions',
'own_gil',
'signals',
]
kwlist = [f'allow_{n}' for n in features]
kwlist[0] = 'use_main_obmalloc'
kwlist[-2] = 'check_multi_interp_extensions'
kwlist[-1] = 'own_gil'
kwlist[-3] = 'check_multi_interp_extensions'
kwlist[-2] = 'own_gil'
kwlist[-1] = 'can_handle_signals'

expected_to_work = {
(True, True, True, True, True, True, True):
(True, True, True, True, True, True, True, True):
(ALL_FLAGS, True),
(True, False, False, False, False, False, False):
(True, False, False, False, False, False, False, False):
(OBMALLOC, False),
(False, False, False, True, False, True, False):
(False, False, False, True, False, True, False, False):
(THREADS | EXTENSIONS, False),
}

expected_to_fail = {
(False, False, False, False, False, False, False),
(False, False, False, False, False, False, False, False),
}

# gh-117649: The free-threaded build does not currently allow
Expand Down Expand Up @@ -1824,14 +1827,16 @@ def test_overridden_setting_extensions_subinterp_check(self):
DAEMON_THREADS = 1<<11
FORK = 1<<15
EXEC = 1<<16
BASE_FLAGS = OBMALLOC | FORK | EXEC | THREADS | DAEMON_THREADS
SIGNALS = 1<<17
BASE_FLAGS = OBMALLOC | FORK | EXEC | THREADS | DAEMON_THREADS | SIGNALS
base_kwargs = {
'use_main_obmalloc': True,
'allow_fork': True,
'allow_exec': True,
'allow_threads': True,
'allow_daemon_threads': True,
'own_gil': False,
'can_handle_signals': True
}

def check(enabled, override):
Expand Down Expand Up @@ -1961,6 +1966,7 @@ class InterpreterConfigTests(unittest.TestCase):
allow_threads=True,
allow_daemon_threads=False,
check_multi_interp_extensions=True,
can_handle_signals=True,
gil='own',
),
'legacy': types.SimpleNamespace(
Expand All @@ -1970,6 +1976,7 @@ class InterpreterConfigTests(unittest.TestCase):
allow_threads=True,
allow_daemon_threads=True,
check_multi_interp_extensions=bool(Py_GIL_DISABLED),
can_handle_signals=False,
gil='shared',
),
'empty': types.SimpleNamespace(
Expand All @@ -1979,6 +1986,7 @@ class InterpreterConfigTests(unittest.TestCase):
allow_threads=False,
allow_daemon_threads=False,
check_multi_interp_extensions=False,
can_handle_signals=False,
gil='default',
),
}
Expand All @@ -1991,16 +1999,18 @@ def iter_all_configs(self):
for allow_threads in (True, False):
for allow_daemon in (True, False):
for checkext in (True, False):
for gil in ('shared', 'own', 'default'):
yield types.SimpleNamespace(
use_main_obmalloc=use_main_obmalloc,
allow_fork=allow_fork,
allow_exec=allow_exec,
allow_threads=allow_threads,
allow_daemon_threads=allow_daemon,
check_multi_interp_extensions=checkext,
gil=gil,
)
for handle_signals in (True, False):
for gil in ('shared', 'own', 'default'):
yield types.SimpleNamespace(
use_main_obmalloc=use_main_obmalloc,
allow_fork=allow_fork,
allow_exec=allow_exec,
allow_threads=allow_threads,
allow_daemon_threads=allow_daemon,
check_multi_interp_extensions=checkext,
can_handle_signals=handle_signals,
gil=gil,
)

def assert_ns_equal(self, ns1, ns2, msg=None):
# This is mostly copied from TestCase.assertDictEqual.
Expand Down Expand Up @@ -2175,6 +2185,7 @@ def new_interp(config):
with self.subTest('main'):
expected = _interpreters.new_config('legacy')
expected.gil = 'own'
expected.can_handle_signals = True
if Py_GIL_DISABLED:
expected.check_multi_interp_extensions = False
interpid, *_ = _interpreters.get_main()
Expand Down
3 changes: 2 additions & 1 deletion Lib/test/test_embed.py
Original file line number Diff line number Diff line change
Expand Up @@ -1827,10 +1827,11 @@ def test_init_main_interpreter_settings(self):
DAEMON_THREADS = 1<<11
FORK = 1<<15
EXEC = 1<<16
SIGNALS = 1<<17
expected = {
# All optional features should be enabled.
'feature_flags':
OBMALLOC | FORK | EXEC | THREADS | DAEMON_THREADS,
OBMALLOC | FORK | EXEC | THREADS | DAEMON_THREADS | SIGNALS,
'own_gil': True,
}
out, err = self.run_embedded_interpreter(
Expand Down
2 changes: 2 additions & 0 deletions Lib/test/test_import/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2194,9 +2194,11 @@ class SubinterpImportTests(unittest.TestCase):
ISOLATED = dict(
use_main_obmalloc=False,
gil=2,
can_handle_signals=True,
)
NOT_ISOLATED = {k: not v for k, v in ISOLATED.items()}
NOT_ISOLATED['gil'] = 1
NOT_ISOLATED['can_handle_signals'] = False

@unittest.skipUnless(hasattr(os, "pipe"), "requires os.pipe()")
def pipe(self):
Expand Down
Loading
Loading