Skip to content

Multiprocessing queue doesn't make the program exit properly #132634

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
albertino87 opened this issue Apr 17, 2025 · 1 comment
Open

Multiprocessing queue doesn't make the program exit properly #132634

albertino87 opened this issue Apr 17, 2025 · 1 comment
Labels
stdlib Python modules in the Lib dir topic-multiprocessing type-bug An unexpected behavior, bug, or error

Comments

@albertino87
Copy link

albertino87 commented Apr 17, 2025 •
edited by github-actions bot
Loading

Bug report

Bug description:

Hi, I'm observing a strange behaviour with python multiprocessing Queue object.

My environment:

OS: Windows 10
python: 3.13.1

but I observed the same with:

OS: Windows 10
python: 3.12.7

and:

OS: Windows 10
python: 3.10.14

I have this short script:

from multiprocessing import Queue

a = list(range(716))
queue: Queue = Queue()
for item in a:
    queue.put(item)
raise ValueError(f"my len: {len(a)}")

If I run it, everything is ok, it raises the error and exits:

Traceback (most recent call last):
  File "C:\Users\uXXXXXX\AppData\Roaming\JetBrains\PyCharmCE2024.3\scratches\scratch_1.py", line 7, in <module>
    raise ValueError(f"my len: {len(a)}")
ValueError: my len: 716

Process finished with exit code 1

but if i change the number from 716 to 717 or any other number above it, it raises the error but doesn't exits, the scripts hangs there. and when i forcefully stop the scripts it exits with code -1

Traceback (most recent call last):
  File "C:\Users\uXXXXXX\AppData\Roaming\JetBrains\PyCharmCE2024.3\scratches\scratch_1.py", line 7, in <module>
    raise ValueError(f"my len: {len(a)}")
ValueError: my len: 717

Process finished with exit code -1

CPython versions tested on:

3.13

Operating systems tested on:

Windows
@albertino87 albertino87 added the type-bug An unexpected behavior, bug, or error label Apr 17, 2025
@ZeroIntensity
Copy link
Member

I would assume it's because the operating system has a preset limit on how much data can be put into a pipe at once, but as such, there's nothing we can really do to fix it. You should read your queue before trying to shut down the program.

@picnixz picnixz added the stdlib Python modules in the Lib dir label Apr 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
stdlib Python modules in the Lib dir topic-multiprocessing type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@picnixz @ZeroIntensity @albertino87