Skip to content

Simplify consumers by making queue.Queue an iterable #120499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
nineteendo opened this issue Jun 14, 2024 · 7 comments
Closed

Simplify consumers by making queue.Queue an iterable #120499

nineteendo opened this issue Jun 14, 2024 · 7 comments
Labels
type-feature A feature request or enhancement

Comments

@nineteendo
Copy link
Contributor

nineteendo commented Jun 14, 2024 •
edited
Loading

Feature or enhancement

Proposal:

class Iterator:
    def __init__(self, queue):
        self.queue = queue

    def __iter__(self):
        return self

    async def __next__(self):
        try:
            return self.queue.get_nowait()
        except (Empty, ShutDown):
            raise StopIteration

class Queue:
    ...
    def __iter__(self):
        return Iterator(self)

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

Linked PRs
@nineteendo nineteendo added the type-feature A feature request or enhancement label Jun 14, 2024
@nineteendo nineteendo changed the title Add queue.Queue.__iter__() Add *.Queue.__iter__() Jun 14, 2024
@bedevere-app bedevere-app bot mentioned this issue Jun 14, 2024
Closed
@sobolevn
Copy link
Member

sobolevn commented Jun 14, 2024 •
edited
Loading

This is a minor feature, which does not need previous discussion elsewhere

Sorry, I might be missing something obvious, but can you please explain a motivation behind this change?

@zware
Copy link
Member

zware commented Jun 14, 2024

@nineteendo, please stop editing your posts after submission. Edit before you submit, and don't submit until you aren't going to change it again. There is not a window in which substantial edits are okay; email notifications will only ever contain the original version, which at this point is much different than what is now here.

@nineteendo
Copy link
Contributor Author

Sorry, I might be missing something obvious, but can you please explain a motivation behind this change?

Guido suggested this here: python/peps#3782 (comment)

Adding __aiter__ seems fairly uncontroversial now we have shutdown. It can just be discussed on Discourse. I'd recommend adding __iter__ to non-asyncio queues.Queue as well, just like we added shutdown to both.

I've made a separate issue, because the title didn't match.

please stop editing your posts after submission

Reverted. I realised we didn't need a separate class, so that's why I use generator in the PR.

Edit before you submit, and don't submit until you aren't going to change it again.

Would you prefer me to create a new issue then?

@sobolevn
Copy link
Member

@nineteendo thank you! Please, next time include this context in the issue :)

@zware
Copy link
Member

zware commented Jun 14, 2024

Would you prefer me to create a new issue then?

No, I would prefer that you stop editing your posts :). Not even to revert previous edits, just no more edits.

My suggestion to edit before submission is intended for any post you make, not just new issues. Make sure you're saying what you want to say on the first try; there's no benefit to saying the wrong thing quickly when you have unlimited time to get it right. If your understanding of something changes, say so in a new message.

@nineteendo
Copy link
Contributor Author

My suggestion to edit before submission is intended for any post you make, not just new issues.

In the future I'll try to create an issue and PR on my own fork first. I don't make significant changes to regular comments (just fixing a typo or suggestion).

@nineteendo
Copy link
Contributor Author

@Zac-HD, multiprocessing.Queue.close() doesn't prevent other processes from putting items on the queue.
So, I'm not sure how we can make this work without a sentinel for every reading process.

@nineteendo nineteendo changed the title Add *.Queue.__iter__() Simplify consumers by making queue.Queue an iterable Jun 21, 2024
@rhettinger rhettinger closed this as not planned Won't fix, can't repro, duplicate, stale Jun 23, 2024
This was referenced Jun 23, 2024
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@rhettinger @sobolevn @zware @nineteendo