Skip to content

bpo-38267: Add thread timeout parameter to loop.shutdown_default_executor() #16360

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
700cf86
Add thread timeout constant
aeros Sep 24, 2019
1e7794b
Add timeout param to `loop.shutdown_default_executor()`
aeros Sep 24, 2019
a262c45
Set timeout param default to None
aeros Sep 24, 2019
b1bb810
Update `loop.shutdown_default_executor() docstring
aeros Sep 24, 2019
8a4c30c
Move constant to runners.py
aeros Sep 24, 2019
ebc1b99
Add thread timeout for `asyncio.run()`
aeros Sep 25, 2019
5b90766
Update `asyncio.run()` docstring for `shutdown_default_executor()`
aeros Sep 25, 2019
f189f04
Update `asyncio.run()` docstring for thread timeout
aeros Sep 25, 2019
16921a1
Update `asyncio.run()` docs
aeros Sep 25, 2019
b678818
Fix whitespace in `asyncio.run()` docstring
aeros Sep 25, 2019
e332257
Update `shutdown_default_executor()` docs
aeros Sep 25, 2019
7f8428b
Patchcheck
aeros Sep 25, 2019
577b2b2
📜🤖 Added by blurb_it.
blurb-it[bot] Sep 25, 2019
b673080
Fix Misc/NEWS Sphinx role
aeros Sep 25, 2019
926de3f
Merge branch 'master' into bpo-38267-shutdown_default_executor-timeout
aeros Oct 1, 2019
f7c623c
Fix documentation for timeout parameter
aeros Oct 3, 2019
169d2d4
Fix documentation for timeout parameter
aeros Oct 3, 2019
f0dde2d
Fix asyncio.run() docs
aeros Oct 3, 2019
b7c0608
Fix loop.shutdown_default_executor() docstring
aeros Oct 3, 2019
1e95afb
Fix loop.shutdown_default_executor docs
aeros Oct 3, 2019
2b1f35d
Fix loop.shutdown_default_executor()
aeros Oct 4, 2019
e549996
Fix asyncio.run() docstring
aeros Oct 4, 2019
8a5121c
Fix runners.py comment
aeros Oct 4, 2019
4264c04
Fix warning
aeros Oct 5, 2019
e9a0f19
Fix RuntimeWarning
aeros Oct 29, 2019
d9c2e37
Fix style nits
gvanrossum Sep 25, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion Doc/library/asyncio-eventloop.rst
Original file line number Diff line number Diff line change
Expand Up @@ -167,13 +167,20 @@ Running and stopping the loop

.. versionadded:: 3.6

.. coroutinemethod:: loop.shutdown_default_executor()
.. coroutinemethod:: loop.shutdown_default_executor(timeout=None)

Schedule the closure of the default executor and wait for it to join all of
the threads in the :class:`ThreadPoolExecutor`. After calling this method, a
:exc:`RuntimeError` will be raised if :meth:`loop.run_in_executor` is called
while using the default executor.

The *timeout* parameter specifies the amount of time the threadpool will
be given to finish joining. The default value is ``None``, which means the
threadpool will be given an indefinite amount of time.

If the timeout duration is reached, a warning is emitted and threadpool is
terminated without waiting for its threads to finish joining.

Note that there is no need to call this function when
:func:`asyncio.run` is used.

Expand Down
4 changes: 4 additions & 0 deletions Doc/library/asyncio-task.rst
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,10 @@ Running an asyncio Program
the end. It should be used as a main entry point for asyncio
programs, and should ideally only be called once.

The threadpool is given a timeout duration of 5 minutes to join its threads.
If the threadpool hasn't finishing joining within that duration, a warning is
emitted and the threadpool is closed.

Example::

async def main():
Expand Down
17 changes: 14 additions & 3 deletions Lib/asyncio/base_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -549,8 +549,13 @@ async def shutdown_asyncgens(self):
'asyncgen': agen
})

async def shutdown_default_executor(self):
"""Schedule the shutdown of the default executor."""
async def shutdown_default_executor(self, timeout=None):
"""Schedule the shutdown of the default executor.

The timeout parameter specifies the amount of time the threadpool will
be given to finish joining. The default value is None, which means
that the threadpool will be given an indefinite amount of time.
"""
self._executor_shutdown_called = True
if self._default_executor is None:
return
Expand All @@ -560,7 +565,13 @@ async def shutdown_default_executor(self):
try:
await future
finally:
thread.join()
thread.join(timeout)

if thread.is_alive():
warnings.warn("The ThreadPoolExecutor did not finishing joining "
f"its threads within {timeout} seconds.",
RuntimeWarning, stacklevel=2)
self._default_executor.shutdown(wait=False)

def _do_shutdown(self, future):
try:
Expand Down
14 changes: 11 additions & 3 deletions Lib/asyncio/runners.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,16 @@
from . import tasks


# Default timeout for joining the threads in the threadpool
THREAD_JOIN_TIMEOUT = 300


def run(main, *, debug=False):
"""Execute the coroutine and return the result.

This function runs the passed coroutine, taking care of
managing the asyncio event loop and finalizing asynchronous
generators.
managing the asyncio event loop, finalizing asynchronous
generators, and closing the threadpool.

This function cannot be called when another asyncio event loop is
running in the same thread.
Expand All @@ -21,6 +25,9 @@ def run(main, *, debug=False):
It should be used as a main entry point for asyncio programs, and should
ideally only be called once.

The threadpool is given a timeout duration of 5 minutes. If its threads
haven't finishing joining within that duration, the threadpool is terminated.

Example:

async def main():
Expand All @@ -45,7 +52,8 @@ async def main():
try:
_cancel_all_tasks(loop)
loop.run_until_complete(loop.shutdown_asyncgens())
loop.run_until_complete(loop.shutdown_default_executor())
loop.run_until_complete(
loop.shutdown_default_executor(timeout=THREAD_JOIN_TIMEOUT))
finally:
events.set_event_loop(None)
loop.close()
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add *timeout* parameter to :meth:`asyncio.loop.shutdown_default_executor`.