Skip to content

Commit 94cc0b9

Browse files
committed
Add doc for buffersize argument of imap and imap_unordered
1 parent dff1167 commit 94cc0b9

File tree

2 files changed

+21
-2
lines changed

2 files changed

+21
-2
lines changed

Doc/library/multiprocessing.rst

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2448,7 +2448,7 @@ with the :class:`Pool` class.
24482448
Callbacks should complete immediately since otherwise the thread which
24492449
handles the results will get blocked.
24502450

2451-
.. method:: imap(func, iterable[, chunksize])
2451+
.. method:: imap(func, iterable[, chunksize[, buffersize]])
24522452

24532453
A lazier version of :meth:`.map`.
24542454

@@ -2462,7 +2462,18 @@ with the :class:`Pool` class.
24622462
``next(timeout)`` will raise :exc:`multiprocessing.TimeoutError` if the
24632463
result cannot be returned within *timeout* seconds.
24642464

2465-
.. method:: imap_unordered(func, iterable[, chunksize])
2465+
The *iterable* is collected immediately rather than lazily, unless a
2466+
*buffersize* is specified to limit the number of submitted tasks whose
2467+
results have not yet been yielded. If the buffer is full, iteration over
2468+
the *iterables* pauses until a result is yielded from the buffer.
2469+
To fully utilize pool's capacity, set *buffersize* to the number of
2470+
processes in pool (to consume *iterable* as you go) or even higher
2471+
(to prefetch *buffersize - processes* arguments).
2472+
2473+
.. versionadded:: 3.15
2474+
Added the *buffersize* parameter.
2475+
2476+
.. method:: imap_unordered(func, iterable[, chunksize[, buffersize]])
24662477

24672478
The same as :meth:`imap` except that the ordering of the results from the
24682479
returned iterator should be considered arbitrary. (Only when there is
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Add the optional ``buffersize`` parameter to
2+
:meth:`multiprocessing.pool.Pool.imap` and
3+
:meth:`multiprocessing.pool.Pool.imap_unordered` to limit the number of
4+
submitted tasks whose results have not yet been yielded. If the buffer is
5+
full, iteration over the *iterables* pauses until a result is yielded from
6+
the buffer. To fully utilize pool's capacity, set *buffersize* to the number
7+
of processes in pool (to consume *iterable* as you go) or even higher (to
8+
prefetch *buffersize - processes* arguments).

0 commit comments

Comments
 (0)