gh-132063: ProcessPoolExecutor swallows falsy Exceptions #132129

YvesDup · 2025-04-05T14:35:57Z

ProcessPoolExecutor and ThreadPoolExecutor swallows falsy exception.

Fix tests about exception variable in 2 files of /Lib/concurrent/futures

_base.py
process.py

Changes 2 tests as below:

Replace if exception: with if exception is not None:

Issue: ProcessPoolExecutor swallows falsy Exceptions #132063

ZeroIntensity · 2025-04-05T16:43:52Z

Misc/NEWS.d/next/Library/2025-04-05-15-05-09.gh-issue-132063.KHnslU.rst

@@ -0,0 +1 @@
+Fix tests of exception attribute in :method:`process_result_item`of :class:`_ExecutorManagerThread` and :method:`__get_result` of :class:`Future` in module :lib:`concurrent`.


Love the effort, but this is too many technical details for a blurb entry.

Suggested change

Fix tests of exception attribute in :method:`process_result_item`of :class:`_ExecutorManagerThread` and :method:`__get_result` of :class:`Future` in module :lib:`concurrent`.

Prevent exceptions that evaluate as falsey from being ignored

by :class:`concurrent.futures.ProcessPoolExecutor`.

This is not limited to __bool__, it can be anything that is falsey, including a class which implements __len__ as well (https://docs.python.org/3/reference/datamodel.html#object.__bool__)

I suggest:

Prevent exceptions, including group exceptions, that evaluate as falsey (With bool and len dunder methods returns) from being ignored
by :class:concurrent.futures.ProcessPoolExecutor and :class:concurrent.futures.ThreadPoolExecutor.

I don't really like to mention bool and len again so how about linking falsey to object.__bool__?

:meth:`falsey <object.__bool__>`

It might be worth adding a "falsey" term to the glossary, but that should be done elsewhere.

Lib/test/test_concurrent_futures/test_process_pool.py

Lib/test/test_concurrent_futures/test_thread_pool.py

python-cla-bot · 2025-04-06T13:55:29Z

All commit authors signed the Contributor License Agreement.

picnixz

I never sent my review.

picnixz · 2025-04-05T17:34:58Z

Misc/NEWS.d/next/Library/2025-04-05-15-05-09.gh-issue-132063.KHnslU.rst

@@ -0,0 +1 @@
+Fix tests of exception attribute in :method:`process_result_item`of :class:`_ExecutorManagerThread` and :method:`__get_result` of :class:`Future` in module :lib:`concurrent`.


This is not limited to __bool__, it can be anything that is falsey, including a class which implements __len__ as well (https://docs.python.org/3/reference/datamodel.html#object.__bool__)

Lib/test/test_concurrent_futures/test_process_pool.py

YvesDup · 2025-04-07T10:22:59Z

After more tests, I found that __cause__, __context__ attributes in falsey exceptions are ignored.

cpython/Lib/traceback.py

Line 1123 in 255eb37

if (e and e.__cause__ is not None

cpython/Lib/traceback.py

Line 1144 in 255eb37

if (e and e.__context__ is not None

And falsey BaseExceptionGroup exception is ignored too.

cpython/Lib/traceback.py

Line 1159 in 255eb37

if e and isinstance(e, BaseExceptionGroup):

Should we open a new issue with these bugs or integrate them to this PR ?

EDIT: please ignore this comment

Lib/test/test_concurrent_futures/executor.py

Misc/NEWS.d/next/Library/2025-04-05-15-05-09.gh-issue-132063.KHnslU.rst

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

picnixz

Some nits and LGTM.

Lib/test/test_concurrent_futures/executor.py

picnixz · 2025-04-08T14:54:20Z

Lib/test/test_concurrent_futures/executor.py

+
+        msg = 'boolbool'
+        with self.assertRaisesRegex(FalseyBoolException, msg):
+            self.executor.submit(raiser, FalseyBoolException, msg).result()


This is for a possible follow-up but this kind of call makes me wonder whether the base class shouldn't have some def submit_and_fetch(self) method that does this.

Does this method return directy the result ?

There is a follow-up with the TracebackExceptionclass which swallows __cause__ and __context__ content of falsey exceptions.
I will post a new issue about this.

miss-islington-app · 2025-04-08T15:11:17Z

Thanks @YvesDup for the PR, and @picnixz for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

…alsey exceptions (pythonGH-132129) (cherry picked from commit 933c665) Co-authored-by: Duprat <yduprat@gmail.com>

bedevere-app · 2025-04-08T15:11:32Z

GH-132275 is a backport of this pull request to the 3.13 branch.

…falsey exceptions (GH-132129) (#132275) gh-132063: ensure that `ProcessPoolExecutor` does not swallow falsey exceptions (GH-132129) (cherry picked from commit 933c665) Co-authored-by: Duprat <yduprat@gmail.com>

…alsey exceptions (python#132129)

Initial commit

9efa479

bedevere-app bot added the awaiting review label Apr 5, 2025

bedevere-app bot mentioned this pull request Apr 5, 2025

ProcessPoolExecutor swallows falsy Exceptions #132063

Closed

blurb-it bot and others added 2 commits April 5, 2025 15:05

📜🤖 Added by blurb_it.

8295a27

Update news

81ec658

ZeroIntensity reviewed Apr 5, 2025

View reviewed changes

picnixz reviewed Apr 6, 2025

View reviewed changes

YvesDup added 3 commits April 8, 2025 12:31

Update news

098ed59

Move test to ExecutorTest and Updates the comment of test

bcf2510

fix nits in news

fbbe501

picnixz reviewed Apr 8, 2025

View reviewed changes

Lib/test/test_concurrent_futures/executor.py Outdated Show resolved Hide resolved

Misc/NEWS.d/next/Library/2025-04-05-15-05-09.gh-issue-132063.KHnslU.rst Outdated Show resolved Hide resolved

YvesDup and others added 3 commits April 8, 2025 14:18

Update news

e450739

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

Add a blank line

ea66f0b

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

Merge branch 'main' into ProcessPoolExecutor-swallows-flasy-exception

5a4e477

picnixz approved these changes Apr 8, 2025

View reviewed changes

Lib/test/test_concurrent_futures/executor.py Show resolved Hide resolved

Lib/test/test_concurrent_futures/executor.py Outdated Show resolved Hide resolved

Lib/test/test_concurrent_futures/executor.py Outdated Show resolved Hide resolved

bedevere-app bot added awaiting merge and removed awaiting review labels Apr 8, 2025

YvesDup added 2 commits April 8, 2025 14:53

Update last reviews and comments

b4de44e

Merge branch 'main' into ProcessPoolExecutor-swallows-flasy-exception

741fc6a

picnixz approved these changes Apr 8, 2025

View reviewed changes

picnixz enabled auto-merge (squash) April 8, 2025 14:55

picnixz added the needs backport to 3.13 bugs and security fixes label Apr 8, 2025

picnixz merged commit 933c665 into python:main Apr 8, 2025
41 checks passed

bedevere-app bot removed the awaiting merge label Apr 8, 2025

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Apr 8, 2025

YvesDup mentioned this pull request Apr 9, 2025

TracebackException swallows attributes of falsey Exception and falsey ExceptionGroup. #132308

Open

seehwan pushed a commit to seehwan/cpython that referenced this pull request Apr 16, 2025

pythongh-132063: ensure that ProcessPoolExecutor does not swallow f…

083adc0

…alsey exceptions (python#132129)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-132063: ProcessPoolExecutor swallows falsy Exceptions #132129

gh-132063: ProcessPoolExecutor swallows falsy Exceptions #132129

YvesDup commented Apr 5, 2025 •

edited by bedevere-app bot

Loading

ZeroIntensity Apr 5, 2025 •

edited by picnixz

Loading

picnixz Apr 5, 2025

YvesDup Apr 7, 2025 •

edited

Loading

picnixz Apr 7, 2025

ZeroIntensity Apr 7, 2025 •

edited

Loading

python-cla-bot bot commented Apr 6, 2025

picnixz left a comment

picnixz Apr 5, 2025

YvesDup commented Apr 7, 2025 •

edited

Loading

picnixz left a comment

picnixz Apr 8, 2025

YvesDup Apr 8, 2025

YvesDup Apr 8, 2025

miss-islington-app bot commented Apr 8, 2025

bedevere-app bot commented Apr 8, 2025

		@@ -0,0 +1 @@
		Fix tests of exception attribute in :method:`process_result_item`of :class:`_ExecutorManagerThread` and :method:`__get_result` of :class:`Future` in module :lib:`concurrent`.

	Fix tests of exception attribute in :method:`process_result_item`of :class:`_ExecutorManagerThread` and :method:`__get_result` of :class:`Future` in module :lib:`concurrent`.
	Prevent exceptions that evaluate as falsey from being ignored
	by :class:`concurrent.futures.ProcessPoolExecutor`.

gh-132063: ProcessPoolExecutor swallows falsy Exceptions #132129

gh-132063: ProcessPoolExecutor swallows falsy Exceptions #132129

Conversation

YvesDup commented Apr 5, 2025 • edited by bedevere-app bot Loading

ZeroIntensity Apr 5, 2025 • edited by picnixz Loading

Choose a reason for hiding this comment

picnixz Apr 5, 2025

Choose a reason for hiding this comment

YvesDup Apr 7, 2025 • edited Loading

Choose a reason for hiding this comment

picnixz Apr 7, 2025

Choose a reason for hiding this comment

ZeroIntensity Apr 7, 2025 • edited Loading

Choose a reason for hiding this comment

python-cla-bot bot commented Apr 6, 2025

picnixz left a comment

Choose a reason for hiding this comment

picnixz Apr 5, 2025

Choose a reason for hiding this comment

YvesDup commented Apr 7, 2025 • edited Loading

picnixz left a comment

Choose a reason for hiding this comment

picnixz Apr 8, 2025

Choose a reason for hiding this comment

YvesDup Apr 8, 2025

Choose a reason for hiding this comment

YvesDup Apr 8, 2025

Choose a reason for hiding this comment

miss-islington-app bot commented Apr 8, 2025

bedevere-app bot commented Apr 8, 2025

YvesDup commented Apr 5, 2025 •

edited by bedevere-app bot

Loading

ZeroIntensity Apr 5, 2025 •

edited by picnixz

Loading

YvesDup Apr 7, 2025 •

edited

Loading

ZeroIntensity Apr 7, 2025 •

edited

Loading

YvesDup commented Apr 7, 2025 •

edited

Loading