Skip to content

gh-115999: Make list and tuple iteration more thread-safe. #128637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Feb 19, 2025

Conversation

Yhg1s
Copy link
Member

@Yhg1s Yhg1s commented Jan 8, 2025

Make tuple iteration more thread-safe, and actually test concurrent iteration of tuple, range and list. (This is prep work for enabling specialization of FOR_ITER in free-threaded builds.) The basic premise is:

  • Iterating over a shared iterable (list, tuple or range) should be safe, not involve data races, and behave like iteration normally does.

  • Using a shared iterator should not crash or involve data races, and should only produce items regular iteration would produce. It is not guaranteed to produce all items, or produce each item only once. (This is not the case for range iteration even after this PR.)

Providing stronger guarantees is possible for some of these iterators, but it's not always straight-forward and can significantly hamper the common case. Since iterators in general aren't shared between threads, and it's simply impossible to concurrently use many iterators (like generators), better to make sharing iterators without explicit synchronization clearly wrong.

Specific issues fixed in order to make the tests pass:

  • List iteration could occasionally fail an assertion when a shared list was shrunk and an item past the new end was retrieved concurrently. There's still some unsafety when deleting/inserting multiple items through for example slice assignment, which uses memmove/memcpy.

  • Tuple iteration could occasionally crash when the iterator's reference to the tuple was cleared on exhaustion. Like with list iteration, in free-threaded builds we can't safely and efficiently clear the iterator's reference to the iterable (doing it safely would mean extra, slow refcount operations), so just keep the iterable reference around.

concurrent iteration. (This is prep work for enabling specialization of
FOR_ITER in free-threaded builds.) The basic premise is:

 - Iterating over a shared _iterable_ (list, tuple or range) should be safe,
   not involve data races, and behave like iteration normally does.

 - Using a shared _iterator_ should not crash or involve data races, and
   should only produce items regular iteration would produce. It is _not_
   guaranteed to produce all items, or produce each item only once.

Providing stronger guarantees is possible for some of these iterators, but
it's not always straight-forward and can significantly hamper the common
case. Since iterators in general aren't shared between threads, and it's
simply impossible to concurrently use many iterators (like generators),
better to make sharing iterators without explicit synchronization clearly
wrong.

Specific issues fixed in order to make the tests pass:

 - List iteration could occasionally crash when a shared list wasn't already
   marked as shared when reallocated.

 - Tuple iteration could occasionally crash when the iterator's reference to
   the tuple was cleared on exhaustion. Like with list iteration, in
   free-threaded builds we can't safely and efficiently clear the iterator's
   reference to the iterable (doing it safely would mean extra, slow
   refcount operations), so just keep the iterable reference around.

 - Fast range iterators (for integers that fit in C longs) shared between
   threads would sometimes produce values past the end of the range, because
   the iterators use two bits of state that we can't efficiently update
   atomically. Rewriting the iterators to have a single bit of state is
   possible, but probably means more math for each iteration and may not be
   worth it.

 - Long range iterators (for other numbers) shared between threads would
   crash catastrophically in a variety of ways. This now uses a critical
   section. Rewriting this to be more efficient is probably possible, but
   since it deals with arbitrary Python objects it's difficult to get right.

There seem to be no more exising races in list_get_item_ref, so drop it from
the tsan suppression list.
@Yhg1s Yhg1s added the skip news label Jan 8, 2025
Yhg1s added 4 commits January 9, 2025 16:10
actually correct and the real problem was an incorrect assert. The fast path
still contains notionally unsafe uses of memcpy/memmove, so add
list_get_item_ref back to the TSan suppressions file.
iterators, and fix build failures because labels can't technically be at the
end of compound statements (a statement must follow the label, even if it's
empty).
Yhg1s added 3 commits January 10, 2025 13:47
…len and

tupleiter_setstate appropriately relaxed.
iterator test to accept the current behaviour of the fast range iterator and
avoid testing the (very unsafe) long range iterator, use threading.Barrier
where appropriate, and add comments to the unsafe uses of memcpy/memmove in
free-threaded builds.
@Yhg1s Yhg1s changed the title gh-115999: Make list, tuple and range iteration more thread-safe. gh-115999: Make list and tuple iteration more thread-safe. Jan 12, 2025
@Yhg1s
Copy link
Member Author

Yhg1s commented Jan 12, 2025

PTAL.

Yhg1s added 3 commits January 20, 2025 15:03
relaxed atomics. (These are not performance-sensitive parts of the code, and
this lets us detect races elsewhere involving the iterators.)
@Yhg1s
Copy link
Member Author

Yhg1s commented Feb 19, 2025

@colesbury Do you still want to look at this?

Copy link
Contributor

@colesbury colesbury left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Yhg1s Yhg1s merged commit 388e1ca into python:main Feb 19, 2025
45 checks passed
@Yhg1s Yhg1s deleted the list-realloc branch February 19, 2025 00:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants