Closed
Description
Bug report
Bug description:
multiprocessing.Pool with spawn method breaks logging.handlers.QueueHandler configuration
import logging
import logging.config
import multiprocessing
import time
def _init(q):
logging.config.dictConfig({
'version': 1,
'disable_existing_loggers': True,
'handlers': {'log_to_parent': {'class': 'logging.handlers.QueueHandler', 'queue': q}},
'root': {'handlers': ['log_to_parent'], 'level': 'DEBUG'}
})
logging.getLogger().info('log from child')
if __name__ == '__main__':
multiprocessing.set_start_method('spawn')
with multiprocessing.Manager() as manager:
# q = manager.Queue()
q = multiprocessing.Queue()
# listen for log messages from child processes
# ...
with multiprocessing.Pool(processes=1, maxtasksperchild=1, initializer=_init, initargs=(q,)) as pool:
time.sleep(1)got exception (when q = manager.Queue() or q = multiprocessing.Queue())
Process SpawnPoolWorker-2:
Traceback (most recent call last):
File "/home/foo/miniconda3/envs/py312forge/lib/python3.12/logging/config.py", line 581, in configure
handler = self.configure_handler(handlers[name])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/foo/miniconda3/envs/py312forge/lib/python3.12/logging/config.py", line 792, in configure_handler
proxy_queue = MM().Queue()
^^^^
File "/home/foo/miniconda3/envs/py312forge/lib/python3.12/multiprocessing/context.py", line 57, in Manager
m.start()
File "/home/foo/miniconda3/envs/py312forge/lib/python3.12/multiprocessing/managers.py", line 562, in start
self._process.start()
File "/home/foo/miniconda3/envs/py312forge/lib/python3.12/multiprocessing/process.py", line 118, in start
assert not _current_process._config.get('daemon'), \
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: daemonic processes are not allowed to have children
The above exception was the direct cause of the following exception:
...
CPython versions tested on:
3.12
Operating systems tested on:
Linux
Linked PRs
- gh-121723: Relax constraints on queue objects for
logging.handlers.QueueHandler. #122154 - [3.13] gh-121723: Relax constraints on queue objects for
logging.handlers.QueueHandler. (GH-122154) #122603 - [3.12] gh-121723: Relax constraints on queue objects for
logging.handlers.QueueHandler. (GH-122154) #122604 - GH-121723: Skip
test_config_queue_handler_multiprocessing_contextin emulated JIT CI #122969 - [3.13] GH-121723: Skip test_config_queue_handler_multiprocessing_context in emulated JIT CI (GH-122969) #122991
Metadata
Metadata
Assignees
Projects
Status
Done