Improve asyncio-dev 'Concurrency and Multithreading' docs #20882
Conversation
|
@aeros Do you have any idea who might be willing to review this change? |
|
I should have time to consider and review these potential changes in the relatively near future. Thanks for the PR. |
|
@aeros Just sending you a friendly ping about this. |
There was a problem hiding this comment.
+1. I'm considering whether or not we want to say "There is no way" or "There is currently no way", because it is very possible that high-level abstraction could be provided over the OS pipes in order to facilitate scheduling callbacks and running coroutines from within a different process, so I'm leaning a bit towards currently. There are not immediate plans to implement something like this, but it's not entirely unfeasible if we can establish clear real-world use cases for something like that (separate from what can already be accomplished with loop.run_in_executor(<PPE_INSTANCE>, func, ...) or the subprocesses API) and if people are interested in working on it.
However, I think this section provides a needed explanation about this topic that doesn't currently exist, so I would be fine with adding it as is. I'll give @1st1 a couple of weeks (from now) to look over it prior to merging though. Feel free to send me another reminder ping if the PR is still open after that without any updates or requested changes. :-)
|
Thanks for the prompt response about this! I changed "There is no way" to "There is currently no way." As I mentioned earlier, this information would have helped me out a lot when I first got started with asyncio. |
|
@aeros Sending you a reminder ping. |
|
Sorry for the delay, I'll proceed with merging this. Just need to close and re-open to restart the CI (it looks like Travis was stalled last time it ran). Thanks again for the PR! |
|
@rogerthat94: Status check is done, and it's a success ✅ . |
|
Sorry, I can't merge this PR. Reason: |
|
Thanks @rogerthat94 for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8, 3.9. |
…0882) I added some information to the `Concurrency and Multithreading` section of the `Developing with asyncio` guide. This is all information that would have helped me when I started using asyncio. I incorrectly assumed that `loop.call_soon_threadsafe()` and `run_coroutine_threadsafe()` could be called from a thread in a process separate from the one that the event loop is running in. Explicitly stating that this will not work will probably help some people starting out with asyncio in the future. I also added references to some other functions that can be used for inter-process communication without blocking the event loop. The section already mentions running blocking code in a ThreadPoolExecutor, but I think listing these other options in this section will also be helpful. (cherry picked from commit c68c5af) Co-authored-by: Roger Iyengar <ri@rogeriyengar.com>
|
GH-22009 is a backport of this pull request to the 3.9 branch. |
…0882) I added some information to the `Concurrency and Multithreading` section of the `Developing with asyncio` guide. This is all information that would have helped me when I started using asyncio. I incorrectly assumed that `loop.call_soon_threadsafe()` and `run_coroutine_threadsafe()` could be called from a thread in a process separate from the one that the event loop is running in. Explicitly stating that this will not work will probably help some people starting out with asyncio in the future. I also added references to some other functions that can be used for inter-process communication without blocking the event loop. The section already mentions running blocking code in a ThreadPoolExecutor, but I think listing these other options in this section will also be helpful. (cherry picked from commit c68c5af) Co-authored-by: Roger Iyengar <ri@rogeriyengar.com>
|
GH-22010 is a backport of this pull request to the 3.8 branch. |
|
@rogerthat94 interesting, this should probably kick off a ticket to make the loop pickle-able, eg to define |
|
@aeros Thank you for all of your help! @graingert Do you want me to create that ticket? Another class that might be worth adding is a version of |
I'd suggest to also start a thread on python-ideas with the proposal of allowing That's not always strictly required for enhancements/new features, but it's a good way to attract attention to it and make a strong argument for why it should be added (and maintained potentially indefinitely in the stdlib). The goal should mainly be to convince @1st1 and/or @asvetlov, since they're the primary experts/maintainers for asyncio. Not that I don't think it would be useful to have, it's more of a question of whether it would be widely used enough to justify supporting it within asyncio (which is primarily targeted at an event loop operating within a single thread, with some interoperability measures to support working with multiple threads). At the moment, the main bridge between multiprocessing and asyncio is really just @rogerthat94
No problem. :-)
The above part about python-ideas also applies to the idea of adding a non-blocking version of |
…0882) (GH-22010) I added some information to the `Concurrency and Multithreading` section of the `Developing with asyncio` guide. This is all information that would have helped me when I started using asyncio. I incorrectly assumed that `loop.call_soon_threadsafe()` and `run_coroutine_threadsafe()` could be called from a thread in a process separate from the one that the event loop is running in. Explicitly stating that this will not work will probably help some people starting out with asyncio in the future. I also added references to some other functions that can be used for inter-process communication without blocking the event loop. The section already mentions running blocking code in a ThreadPoolExecutor, but I think listing these other options in this section will also be helpful. (cherry picked from commit c68c5af) Co-authored-by: Roger Iyengar <ri@rogeriyengar.com>
…0882) (GH-22009) I added some information to the `Concurrency and Multithreading` section of the `Developing with asyncio` guide. This is all information that would have helped me when I started using asyncio. I incorrectly assumed that `loop.call_soon_threadsafe()` and `run_coroutine_threadsafe()` could be called from a thread in a process separate from the one that the event loop is running in. Explicitly stating that this will not work will probably help some people starting out with asyncio in the future. I also added references to some other functions that can be used for inter-process communication without blocking the event loop. The section already mentions running blocking code in a ThreadPoolExecutor, but I think listing these other options in this section will also be helpful. (cherry picked from commit c68c5af) Co-authored-by: Roger Iyengar <ri@rogeriyengar.com>
…0882) I added some information to the `Concurrency and Multithreading` section of the `Developing with asyncio` guide. This is all information that would have helped me when I started using asyncio. I incorrectly assumed that `loop.call_soon_threadsafe()` and `run_coroutine_threadsafe()` could be called from a thread in a process separate from the one that the event loop is running in. Explicitly stating that this will not work will probably help some people starting out with asyncio in the future. I also added references to some other functions that can be used for inter-process communication without blocking the event loop. The section already mentions running blocking code in a ThreadPoolExecutor, but I think listing these other options in this section will also be helpful.
I added some information to the
Concurrency and Multithreadingsection of theDeveloping with asyncioguide.This is all information that would have helped me when I started using asyncio. I incorrectly assumed that
loop.call_soon_threadsafe()andrun_coroutine_threadsafe()could be called from a thread in a process separate from the one that the event loop is running in. Explicitly stating that this will not work will probably help some people starting out with asyncio in the future.I also added references to some other functions that can be used for inter-process communication without blocking the event loop. The section already mentions running blocking code in a ThreadPoolExecutor, but I think listing these other options in this section will also be helpful.
Automerge-Triggered-By: @aeros