Skip to content

Commit fc37cb4

Browse files
committed
Add doc for buffersize argument of imap and imap_unordered
1 parent c6e6074 commit fc37cb4

File tree

3 files changed

+29
-1
lines changed

3 files changed

+29
-1
lines changed

Doc/library/multiprocessing.rst

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2408,7 +2408,7 @@ with the :class:`Pool` class.
24082408
Callbacks should complete immediately since otherwise the thread which
24092409
handles the results will get blocked.
24102410

2411-
.. method:: imap(func, iterable[, chunksize])
2411+
.. method:: imap(func, iterable[, chunksize[, buffersize]])
24122412

24132413
A lazier version of :meth:`.map`.
24142414

@@ -2422,6 +2422,17 @@ with the :class:`Pool` class.
24222422
``next(timeout)`` will raise :exc:`multiprocessing.TimeoutError` if the
24232423
result cannot be returned within *timeout* seconds.
24242424

2425+
The *iterable* is collected immediately rather than lazily, unless a
2426+
*buffersize* is specified to limit the number of submitted tasks whose
2427+
results have not yet been yielded. If the buffer is full, iteration over
2428+
the *iterables* pauses until a result is yielded from the buffer.
2429+
To fully utilize pool's capacity, set *buffersize* to the number of
2430+
processes in pool (to consume *iterable* as you go) or even higher
2431+
(to prefetch *buffersize - processes* arguments).
2432+
2433+
.. versionadded:: 3.15
2434+
Added the *buffersize* parameter.
2435+
24252436
.. method:: imap_unordered(func, iterable[, chunksize])
24262437

24272438
The same as :meth:`imap` except that the ordering of the results from the
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
Add the optional ``buffersize`` parameter to
2+
:meth:`multiprocessing.pool.Pool.imap` and
3+
:meth:`multiprocessing.pool.Pool.imap_unordered` to limit the number of
4+
submitted tasks whose results have not yet been yielded. If the buffer is full,
5+
iteration over the *iterables* pauses until a result is yielded from the buffer.
6+
7+
To fully utilize pool's capacity, set *buffersize* to the number of
8+
processes in pool (to consume *iterable* as you go) or even higher
9+
(to prefetch *buffersize - processes* arguments).
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Add the optional ``buffersize`` parameter to
2+
:meth:`multiprocessing.pool.Pool.imap` and
3+
:meth:`multiprocessing.pool.Pool.imap_unordered` to limit the number of
4+
submitted tasks whose results have not yet been yielded. If the buffer is
5+
full, iteration over the *iterables* pauses until a result is yielded from
6+
the buffer. To fully utilize pool's capacity, set *buffersize* to the number
7+
of processes in pool (to consume *iterable* as you go) or even higher (to
8+
prefetch *buffersize - processes* arguments).

0 commit comments

Comments
 (0)