Skip to content

bpo-41229: Update docs for explicit aclose()-required cases and add contextlib.aclosing() method #21545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Nov 2, 2020
33 changes: 33 additions & 0 deletions Doc/library/contextlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,39 @@ Functions and classes provided:
``page.close()`` will be called when the :keyword:`with` block is exited.


.. class:: aclosing(thing)

Return an async context manager that calls the ``aclose()`` method of *thing*
upon completion of the block. This is basically equivalent to::

from contextlib import asynccontextmanager

@asynccontextmanager
async def aclosing(thing):
try:
yield thing
finally:
await thing.aclose()

Significantly, ``aclosing()`` supports deterministic cleanup of async
generators when they happen to exit early by :keyword:`break` or an
exception. For example::

from contextlib import aclosing

async with aclosing(my_generator()) as values:
async for value in values:
if value == 42:
break

This pattern ensures that the generator's async exit code is executed in
the same context as its iterations (so that exceptions and context
variables work as expected, and the exit code isn't run after the
lifetime of some task it depends on).

.. versionadded:: 3.10


.. _simplifying-support-for-single-optional-context-managers:

.. function:: nullcontext(enter_result=None)
Expand Down
16 changes: 13 additions & 3 deletions Doc/reference/expressions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -639,6 +639,16 @@ after resuming depends on the method which resumed the execution. If
:meth:`~agen.asend` is used, then the result will be the value passed in to
that method.

If an asynchronous generator happens to exit early by :keyword:`break`, the caller
task being cancelled, or other exceptions, the generator's async cleanup code
will run and possibly raise exceptions or access context variables in an
unexpected context--perhaps after the lifetime of tasks it depends, or
during the event loop shutdown when the async-generator garbage collection hook
is called.
To prevent this, the caller must explicitly close the async generator by calling
:meth:`~agen.aclose` method to finalize the generator and ultimately detach it
from the event loop.

In an asynchronous generator function, yield expressions are allowed anywhere
in a :keyword:`try` construct. However, if an asynchronous generator is not
resumed before it is finalized (by reaching a zero reference count or by
Expand All @@ -650,9 +660,9 @@ generator-iterator's :meth:`~agen.aclose` method and run the resulting
coroutine object, thus allowing any pending :keyword:`!finally` clauses
to execute.

To take care of finalization, an event loop should define
a *finalizer* function which takes an asynchronous generator-iterator
and presumably calls :meth:`~agen.aclose` and executes the coroutine.
To take care of finalization upon event loop termination, an event loop should
define a *finalizer* function which takes an asynchronous generator-iterator and
presumably calls :meth:`~agen.aclose` and executes the coroutine.
This *finalizer* may be registered by calling :func:`sys.set_asyncgen_hooks`.
When first iterated over, an asynchronous generator-iterator will store the
registered *finalizer* to be called upon finalization. For a reference example
Expand Down
26 changes: 26 additions & 0 deletions Lib/contextlib.py
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,32 @@ def __exit__(self, *exc_info):
self.thing.close()


class aclosing(AbstractAsyncContextManager):
"""Async context manager for safely finalizing an asynchronously cleaned-up
resource such as an async generator, calling its ``aclose()`` method.

Code like this:

async with aclosing(<module>.fetch(<arguments>)) as agen:
<block>

is equivalent to this:

agen = <module>.fetch(<arguments>)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

await fetch()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed!

try:
<block>
finally:
await agen.aclose()

"""
def __init__(self, thing):
self.thing = thing
async def __aenter__(self):
return self.thing
async def __aexit__(self, *exc_info):
await self.thing.aclose()


class _RedirectStream(AbstractContextManager):

_stream = None
Expand Down
59 changes: 58 additions & 1 deletion Lib/test/test_contextlib_async.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import asyncio
from contextlib import asynccontextmanager, AbstractAsyncContextManager, AsyncExitStack
from contextlib import aclosing, asynccontextmanager, AbstractAsyncContextManager, AsyncExitStack
import functools
from test import support
import unittest
Expand Down Expand Up @@ -279,6 +279,63 @@ async def woohoo(self, func, args, kwds):
self.assertEqual(target, (11, 22, 33, 44))


class AclosingTestCase(unittest.TestCase):

@support.requires_docstrings
def test_instance_docs(self):
cm_docstring = aclosing.__doc__
obj = aclosing(None)
self.assertEqual(obj.__doc__, cm_docstring)

@_async_test
async def test_aclosing(self):
state = []
class C:
async def aclose(self):
state.append(1)
x = C()
self.assertEqual(state, [])
async with aclosing(x) as y:
self.assertEqual(x, y)
self.assertEqual(state, [1])

@_async_test
async def test_aclosing_error(self):
state = []
class C:
async def aclose(self):
state.append(1)
x = C()
self.assertEqual(state, [])
with self.assertRaises(ZeroDivisionError):
async with aclosing(x) as y:
self.assertEqual(x, y)
1 / 0
self.assertEqual(state, [1])

@_async_test
async def test_aclosing_bpo41229(self):
state = []

class Resource:
def __del__(self):
state.append(1)

async def agenfunc():
r = Resource()
yield -1
yield -2

x = agenfunc()
self.assertEqual(state, [])
with self.assertRaises(ZeroDivisionError):
async with aclosing(x) as y:
self.assertEqual(x, y)
self.assertEqual(-1, await x.__anext__())
1 / 0
self.assertEqual(state, [1])


class TestAsyncExitStack(TestBaseExitStack, unittest.TestCase):
class SyncAsyncExitStack(AsyncExitStack):
@staticmethod
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Add ``contextlib.aclosing`` for deterministic cleanup of async generators
which is analogous to ``contextlib.closing`` for non-async generators.
Patch by Joongi Kim and John Belmonte.