BUG: Fix reference counting for subarrays containing objects #12781
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…2650) * BUG: Fix dtype object subarrays for INCREF/DECREF and zeros/empty init * BUG: VOID_copyswap cannot fast path nontrivial subarrays Especially object dtypes need reference counting, so cannot use memcpy. Subarrays that include fields also have data which should not be overwritten. * BUG: Copying subarrays with objects must decref original objects When not using the dtype transfer function, the buffers are initialized to NULL, and then just copied in full. Thus, at no point decref'ing happens on the data that is originally in the array. * TST: Test subarray field reference counting Tests initialization, copying and repeating (incrementing) as well as advanced indexing (uses copyswap currently) and assignment * TST: Test sparse subarray field assignments Assigning to subarray fields should not change the output in between the fields. This currently still happens for normal assignment, but not for advanced indexing. The second test is marked as `pytest.mark.valgrind_error` to make it clear that it is expected to show errors in valgrind, and give at least the theoretical the possibility to skip it during testing.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of #12650.
BUG: Fix dtype object subarrays for INCREF/DECREF and zeros/empty init
BUG: VOID_copyswap cannot fast path nontrivial subarrays
Especially object dtypes need reference counting, so cannot use
memcpy. Subarrays that include fields also have data which should
not be overwritten.
When not using the dtype transfer function, the buffers are initialized
to NULL, and then just copied in full. Thus, at no point decref'ing
happens on the data that is originally in the array.
Tests initialization, copying and repeating (incrementing) as
well as advanced indexing (uses copyswap currently) and assignment
Assigning to subarray fields should not change the output in
between the fields. This currently still happens for normal
assignment, but not for advanced indexing.
The second test is marked as
pytest.mark.valgrind_errorto makeit clear that it is expected to show errors in valgrind, and give
at least the theoretical the possibility to skip it during testing.