bpo-33649: More improvements (pythonGH-9439)

miss-islington · 1st1 · web-flow · commit 8e5ef58c10a1 · 2018-09-20T09:57:19.000-07:00
(cherry picked from commit e247b46) Co-authored-by: Yury Selivanov <yury@magic.io>
diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst
@@ -749,7 +749,7 @@ Watching file descriptors
    invoke *callback* with the specified arguments once *fd* is available for
    writing.
 
-   Use :func:`functools.partial` :ref:`to pass keywords
+   Use :func:`functools.partial` :ref:`to pass keyword arguments
    <asyncio-pass-keywords>` to *func*.
 
 .. method:: loop.remove_writer(fd)
@@ -963,7 +963,7 @@ Unix signals
    Raise :exc:`ValueError` if the signal number is invalid or uncatchable.
    Raise :exc:`RuntimeError` if there is a problem setting up the handler.
 
-   Use :func:`functools.partial` :ref:`to pass keywords
+   Use :func:`functools.partial` :ref:`to pass keyword arguments
    <asyncio-pass-keywords>` to *func*.
 
 .. method:: loop.remove_signal_handler(sig)
@@ -990,11 +990,52 @@ Executing code in thread or process pools
    The *executor* argument should be an :class:`concurrent.futures.Executor`
    instance. The default executor is used if *executor* is ``None``.
 
-   Use :func:`functools.partial` :ref:`to pass keywords
-   <asyncio-pass-keywords>` to *func*.
+   Example::
+
+      import asyncio
+      import concurrent.futures
+
+      def blocking_io():
+          # File operations (such as logging) can block the
+          # event loop: run them in a thread pool.
+          with open('/dev/urandom', 'rb') as f:
+              return f.read(100)
+
+      def cpu_bound():
+          # CPU-bound operations will block the event loop:
+          # in general it is preferable to run them in a
+          # process pool.
+          return sum(i * i for i in range(10 ** 7))
+
+      async def main():
+          loop = asyncio.get_running_loop()
+
+          ## Options:
+
+          # 1. Run in the default loop's executor:
+          result = await loop.run_in_executor(
+              None, blocking_io)
+          print('default thread pool', result)
+
+          # 2. Run in a custom thread pool:
+          with concurrent.futures.ThreadPoolExecutor() as pool:
+              result = await loop.run_in_executor(
+                  pool, blocking_io)
+              print('custom thread pool', result)
+
+          # 3. Run in a custom process pool:
+          with concurrent.futures.ProcessPoolExecutor() as pool:
+              result = await loop.run_in_executor(
+                  pool, cpu_bound)
+              print('custom process pool', result)
+
+      asyncio.run(main())
 
    This method returns a :class:`asyncio.Future` object.
 
+   Use :func:`functools.partial` :ref:`to pass keyword arguments
+   <asyncio-pass-keywords>` to *func*.
+
    .. versionchanged:: 3.5.3
       :meth:`loop.run_in_executor` no longer configures the
       ``max_workers`` of the thread pool executor it creates, instead
diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst
@@ -109,50 +109,89 @@ To actually run a coroutine asyncio provides three main mechanisms:
 Awaitables
 ==========
 
-We say that an object is an *awaitable* object if it can be used
-in an :keyword:`await` expression.
+We say that an object is an **awaitable** object if it can be used
+in an :keyword:`await` expression.  Many asyncio APIs are designed to
+accept awaitables.
 
+There are three main types of *awaitable* objects:
+**coroutines**, **Tasks**, and **Futures**.
 
-.. rubric:: Coroutines and Tasks
 
-Python coroutines are *awaitables*::
+.. rubric:: Coroutines
+
+Python coroutines are *awaitables* and therefore can be awaited from
+other coroutines::
+
+    import asyncio
 
     async def nested():
         return 42
 
     async def main():
-        # Will print "42":
-        print(await nested())
+        # Nothing happens if we just call "nested()".
+        # (a coroutine object is created but not awaited)
+        nested()
+
+        # Let's do it differently now and await it:
+        print(await nested())  # will print "42".
+
+    asyncio.run(main())
+
+.. important::
+
+   In this documentation the term "coroutine" can be used for
+   two closely related concepts:
+
+   * a *coroutine function*: an :keyword:`async def` function;
+
+   * a *coroutine object*: an object returned by calling a
+     *coroutine function*.
+
+asyncio also supports legacy :ref:`generator-based
+<asyncio_generator_based_coro>` coroutines.
+
+
+.. rubric:: Tasks
 
 *Tasks* are used to schedule coroutines *concurrently*.
-See the previous :ref:`section <coroutine>` for an introduction
-to coroutines and tasks.
 
-Note that in this documentation the term "coroutine" can be used for
-two closely related concepts:
+When a coroutine is wrapped into a *Task* with functions like
+:func:`asyncio.create_task` the coroutine is automatically
+scheduled to run soon::
+
+    import asyncio
+
+    async def nested():
+        return 42
 
-* a *coroutine function*: an :keyword:`async def` function;
+    async def main():
+        # Schedule nested() to run soon concurrently
+        # with "main()".
+        task = asyncio.create_task(nested())
 
-* a *coroutine object*: object returned by calling a
-  *coroutine function*.
+        # "task" can now be used to cancel "nested()", or
+        # can simply be awaited to wait until it is complete:
+        await task
+
+    asyncio.run(main())
 
 
 .. rubric:: Futures
 
-There is a dedicated section about the :ref:`asyncio Future object
-<asyncio-futures>`, but the concept is fundamental to asyncio so
-it needs a brief introduction in this section.
+A :class:`Future` is a special **low-level** awaitable object that
+represents an **eventual result** of an asynchronous operation.
+
+When a Future object is *awaited* it means that the coroutine will
+wait until the Future is resolved in some other place.
 
-A Future is a special **low-level** awaitable object that represents
-an **eventual result** of an asynchronous operation.
 Future objects in asyncio are needed to allow callback-based code
 to be used with async/await.
 
-Normally, **there is no need** to create Future objects at the
+Normally **there is no need** to create Future objects at the
 application level code.
 
 Future objects, sometimes exposed by libraries and some asyncio
-APIs, should be awaited::
+APIs, can be awaited::
 
     async def main():
         await function_that_returns_a_future_object()
@@ -163,6 +202,9 @@ APIs, should be awaited::
             some_python_coroutine()
         )
 
+A good example of a low-level function that returns a Future object
+is :meth:`loop.run_in_executor`.
+
 
 Running an asyncio Program
 ==========================
@@ -192,8 +234,8 @@ Creating Tasks
 
 .. function:: create_task(coro)
 
-   Wrap the *coro* :ref:`coroutine <coroutine>` into a Task and
-   schedule its execution.  Return the Task object.
+   Wrap the *coro* :ref:`coroutine <coroutine>` into a :class:`Task`
+   and schedule its execution.  Return the Task object.
 
    The task is executed in the loop returned by :func:`get_running_loop`,
    :exc:`RuntimeError` is raised if there is no running loop in
@@ -253,17 +295,17 @@ Sleeping
 Running Tasks Concurrently
 ==========================
 
-.. awaitablefunction:: gather(\*fs, loop=None, return_exceptions=False)
+.. awaitablefunction:: gather(\*aws, loop=None, return_exceptions=False)
 
-   Run :ref:`awaitable objects <asyncio-awaitables>` in the *fs*
+   Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
    sequence *concurrently*.
 
-   If any awaitable in *fs* is a coroutine, it is automatically
+   If any awaitable in *aws* is a coroutine, it is automatically
    scheduled as a Task.
 
    If all awaitables are completed successfully, the result is an
    aggregate list of returned values.  The order of result values
-   corresponds to the order of awaitables in *fs*.
+   corresponds to the order of awaitables in *aws*.
 
    If *return_exceptions* is ``True``, exceptions are treated the
    same as successful results, and aggregated in the result list.
@@ -273,7 +315,7 @@ Running Tasks Concurrently
    If ``gather`` is *cancelled*, all submitted awaitables
    (that have not completed yet) are also *cancelled*.
 
-   If any Task or Future from the *fs* sequence is *cancelled*, it is
+   If any Task or Future from the *aws* sequence is *cancelled*, it is
    treated as if it raised :exc:`CancelledError` -- the ``gather()``
    call is **not** cancelled in this case.  This is to prevent the
    cancellation of one submitted Task/Future to cause other
@@ -323,13 +365,13 @@ Running Tasks Concurrently
 Shielding Tasks From Cancellation
 =================================
 
-.. awaitablefunction:: shield(fut, \*, loop=None)
+.. awaitablefunction:: shield(aw, \*, loop=None)
 
    Protect an :ref:`awaitable object <asyncio-awaitables>`
    from being :meth:`cancelled <Task.cancel>`.
 
-   *fut* can be a coroutine, a Task, or a Future-like object.  If
-   *fut* is a coroutine it is automatically scheduled as a Task.
+   *aw* can be a coroutine, a Task, or a Future-like object.  If
+   *aw* is a coroutine it is automatically scheduled as a Task.
 
    The statement::
 
@@ -361,12 +403,12 @@ Shielding Tasks From Cancellation
 Timeouts
 ========
 
-.. coroutinefunction:: wait_for(fut, timeout, \*, loop=None)
+.. coroutinefunction:: wait_for(aw, timeout, \*, loop=None)
 
-   Wait for the *fut* :ref:`awaitable <asyncio-awaitables>`
+   Wait for the *aw* :ref:`awaitable <asyncio-awaitables>`
    to complete with a timeout.
 
-   If *fut* is a coroutine it is automatically scheduled as a Task.
+   If *aw* is a coroutine it is automatically scheduled as a Task.
 
    *timeout* can either be ``None`` or a float or int number of seconds
    to wait for.  If *timeout* is ``None``, block until the future
@@ -381,7 +423,7 @@ Timeouts
    The function will wait until the future is actually cancelled,
    so the total wait time may exceed the *timeout*.
 
-   If the wait is cancelled, the future *fut* is also cancelled.
+   If the wait is cancelled, the future *aw* is also cancelled.
 
    The *loop* argument is deprecated and scheduled for removal
    in Python 4.0.
@@ -409,22 +451,22 @@ Timeouts
        #     timeout!
 
    .. versionchanged:: 3.7
-      When *fut* is cancelled due to a timeout, ``wait_for`` waits
-      for *fut* to be cancelled.  Previously, it raised
+      When *aw* is cancelled due to a timeout, ``wait_for`` waits
+      for *aw* to be cancelled.  Previously, it raised
       :exc:`asyncio.TimeoutError` immediately.
 
 
 Waiting Primitives
 ==================
 
-.. coroutinefunction:: wait(fs, \*, loop=None, timeout=None,\
+.. coroutinefunction:: wait(aws, \*, loop=None, timeout=None,\
                             return_when=ALL_COMPLETED)
 
-   Run :ref:`awaitable objects <asyncio-awaitables>` in the *fs*
+   Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
    sequence concurrently and block until the condition specified
    by *return_when*.
 
-   If any awaitable in *fs* is a coroutine, it is automatically
+   If any awaitable in *aws* is a coroutine, it is automatically
    scheduled as a Task.
 
    Returns two sets of Tasks/Futures: ``(done, pending)``.
@@ -465,12 +507,12 @@ Waiting Primitives
 
    Usage::
 
-        done, pending = await asyncio.wait(fs)
+        done, pending = await asyncio.wait(aws)
 
 
-.. function:: as_completed(fs, \*, loop=None, timeout=None)
+.. function:: as_completed(aws, \*, loop=None, timeout=None)
 
-   Run :ref:`awaitable objects <asyncio-awaitables>` in the *fs*
+   Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
    set concurrently.  Return an iterator of :class:`Future` objects.
    Each Future object returned represents the earliest result
    from the set of the remaining awaitables.
@@ -480,7 +522,7 @@ Waiting Primitives
 
    Example::
 
-       for f in as_completed(fs):
+       for f in as_completed(aws):
            earliest_result = await f
            # ...
 
@@ -670,6 +712,52 @@ Task Object
       A Task is *done* when the wrapped coroutine either returned
       a value, raised an exception, or the Task was cancelled.
 
+   .. method:: result()
+
+      Return the result of the Task.
+
+      If the Task is *done*, the result of the wrapped coroutine
+      is returned (or if the coroutine raised an exception, that
+      exception is re-raised.)
+
+      If the Task has been *cancelled*, this method raises
+      a :exc:`CancelledError` exception.
+
+      If the Task's result isn't yet available, this method raises
+      a :exc:`InvalidStateError` exception.
+
+   .. method:: exception()
+
+      Return the exception of the Task.
+
+      If the wrapped coroutine raised an exception that exception
+      is returned.  If the wrapped coroutine returned normally
+      this method returns ``None``.
+
+      If the Task has been *cancelled*, this method raises a
+      :exc:`CancelledError` exception.
+
+      If the Task isn't *done* yet, this method raises an
+      :exc:`InvalidStateError` exception.
+
+   .. method:: add_done_callback(callback, *, context=None)
+
+      Add a callback to be run when the Task is *done*.
+
+      This method should only be used in low-level callback-based code.
+
+      See the documentation of :meth:`Future.add_done_callback`
+      for more details.
+
+   .. method:: remove_done_callback(callback)
+
+      Remove *callback* from the callbacks list.
+
+      This method should only be used in low-level callback-based code.
+
+      See the documentation of :meth:`Future.remove_done_callback`
+      for more details.
+
    .. method:: get_stack(\*, limit=None)
 
       Return the list of stack frames for this Task.
