From 2f05972013837e95debf3c53ee4fe4ef65be4242 Mon Sep 17 00:00:00 2001 From: Lucas Wiman Date: Sun, 19 Jun 2022 11:22:33 -0700 Subject: [PATCH 1/2] Add warning to ThreadPoolExecutor docs See https://github.com/python/cpython/issues/86128#issuecomment-1159332994 for an example where this happens. Essentially, even if it looks like you added an `atexit` handler to instruct your thread to exit gracefully, it will only be executed _after_ your thread has finished. For long-running threads (e.g. threads listening to a queue), that may never happen at all. Elsewhere in #86128, it's recommended that `ThreadPoolExecutor` not be used for long-running tasks, but this was not reflected in the documentation. Based solely on the API, there is no reason to think you shouldn't use it for long-running tasks. The only reason appears to be a limitation in its implementation, so that should be made explicit in the docs. --- Doc/library/concurrent.futures.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst index 99703ff3051d47..205980233afaf5 100644 --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -149,6 +149,13 @@ And:: An :class:`Executor` subclass that uses a pool of at most *max_workers* threads to execute calls asynchronously. + All threads enqueued to ``ThreadPoolExecutor`` will be joined before the + interpreter can exit. Note that the exit handler which does this is + executed *before* any exit handlers added using `atexit`. This means + exceptions in the main thread must be caught and handled in order to + signal threads to exit gracefully. For this reason, it is recommended + that ``ThreadPoolExecutor`` not be used for long-running tasks. + *initializer* is an optional callable that is called at the start of each worker thread; *initargs* is a tuple of arguments passed to the initializer. Should *initializer* raise an exception, all currently From 2bde13231a435eddb9ceb0a12753b4a2db39cb6e Mon Sep 17 00:00:00 2001 From: "blurb-it[bot]" <43283697+blurb-it[bot]@users.noreply.github.com> Date: Sun, 19 Jun 2022 18:18:23 +0000 Subject: [PATCH 2/2] =?UTF-8?q?=F0=9F=93=9C=F0=9F=A4=96=20Added=20by=20blu?= =?UTF-8?q?rb=5Fit.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../Documentation/2022-06-19-18-18-22.gh-issue-86128.39DDTD.rst | 1 + 1 file changed, 1 insertion(+) create mode 100644 Misc/NEWS.d/next/Documentation/2022-06-19-18-18-22.gh-issue-86128.39DDTD.rst diff --git a/Misc/NEWS.d/next/Documentation/2022-06-19-18-18-22.gh-issue-86128.39DDTD.rst b/Misc/NEWS.d/next/Documentation/2022-06-19-18-18-22.gh-issue-86128.39DDTD.rst new file mode 100644 index 00000000000000..bab006856deea7 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2022-06-19-18-18-22.gh-issue-86128.39DDTD.rst @@ -0,0 +1 @@ +Document a limitation in ThreadPoolExecutor where its exit handler is executed before any handlers in atexit.