Skip to content

Bug in masked array indexing. #7493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
kawing-chiu opened this issue Apr 1, 2016 · 8 comments · Fixed by #7659
Closed

Bug in masked array indexing. #7493

kawing-chiu opened this issue Apr 1, 2016 · 8 comments · Fixed by #7659
Labels
00 - Bug component: numpy.ma masked arrays

Comments

@kawing-chiu
Copy link

This simple indexing operation

ma = np.ma.zeros(5000, dtype=[('symbol', 'O')])
print(ma[0])

fails with a

ValueError: Setting void-array with object members using buffer.

exception. While the non-masked version just works:

arr = np.zeros(5000, dtype=[('symbol', 'O')])
print(arr[0])

Numpy version 1.11.0 on python 3.5.1 on arch linux.
@ahaldane
Copy link
Member

Probably caused by #6094.

@charris charris added this to the 1.11.1 release milestone May 11, 2016
@charris
Copy link
Member

charris commented May 14, 2016

@ahaldane Does this need a fix?

@ahaldane
Copy link
Member

It is a regression, but the question of what to do is a bit complicated. Here are the facts:

I am tempted to leave this alone until we get around to considering #6053. #6094 fixed more cases than it broke: Now all structured dtypes work properly, except structured dtypes with objects.

Here is a demonstration of the assignment bug:

arr = np.array([(0,), (1,)], dtype=[('f', 'O')])
arr[0] = arr[1]
ValueError: Setting void-array with object members using buffer.

Maybe @astrofrog has more insights into a quick fix?

@ahaldane
Copy link
Member

Hmm actually there is a hack to get around the assignment bug:

>>> arr = np.array([(0,), (1,)], dtype=[('f', 'O')])
>>> arr[0] = arr[1]
ValueError: Setting void-array with object members using buffer.
>>> arr[:1] = arr[1]  #works because it goes down the dtype_transfer code path

That means a very hacky fix to #6094 may be to replace res = np.array([m._data], dtype=rdtype) with

res = np.empty(1, rdtype)
res[:1] = m._data

@charris
Copy link
Member

charris commented May 22, 2016

@ahaldane Will #6053 fix this, 0r is it just a good starting point?

@ahaldane
Copy link
Member

I just did a quick rebase of #6053 on master to make sure, and it fixes it with no other changes needed.

@charris
Copy link
Member

charris commented May 22, 2016

#6053 is bigger than I want to contemplate for 1.11.1. If you think your workaround is reasonable, could you make a special PR against 1.11.x? If not, I'm inclined to just let this problem be for 1.11 and shoot for 1.12 for the fix.

@ahaldane ahaldane mentioned this issue May 22, 2016
Merged
@ahaldane
Copy link
Member

Sure, as long as we remember to change it later. I'll try it out.

ahaldane added a commit to ahaldane/numpy that referenced this issue May 22, 2016
@ahaldane
BUG: Temporary fix for str(mvoid) for object field types
7a05c91 
Fixes numpy#7493, using a temporary hack, to be properly fixed later
(eg with numpy#6053)

Printing a Masked-Void instance broke if the instance has a field of
Object dtype because assignment involving structured dtypes with objects
doesn't work. Fix is to use dtype-transfer code which avoid the bug.
@ahaldane ahaldane mentioned this issue May 22, 2016
Merged
ahaldane added a commit to ahaldane/numpy that referenced this issue May 22, 2016
@ahaldane
BUG: Temporary fix for str(mvoid) for object field types
a48f422 
Fixes numpy#7493, using a temporary hack, to be properly fixed later
(eg with numpy#6053)

Printing a Masked-Void instance broke if the instance has a field of
Object dtype because assignment involving structured dtypes with objects
doesn't work. Fix is to use dtype-transfer code which avoid the bug.
@ahaldane ahaldane mentioned this issue May 22, 2016
Merged
ahaldane added a commit to ahaldane/numpy that referenced this issue Jun 17, 2016
@ahaldane
MAINT: improve struct assignment, allow multi fields views
2ef0103 
This commit attempts to make structure assignment more consistent, and
then changes multi-field indices to return a view instead of a copy.

Assignment between structures now works "by field position" rather than
"by field name".

Fixes numpy#2353, fixes numpy#6085, fixes numpy#3351, fixes numpy#6085, fixes numpy#6314,
fixes numpy#2346, fixes numpy#7058, fixes numpy#3641, fixes numpy#5994, fixes numpy#7262,
fixes numpy#7493
ahaldane added a commit to ahaldane/numpy that referenced this issue Sep 7, 2017
@ahaldane
MAINT: improve struct assignment, allow multi fields views
7412b85 
This commit attempts to make structure assignment more consistent, and
then changes multi-field indices to return a view instead of a copy.

Assignment between structures now works "by field position" rather than
"by field name".

Fixes numpy#2353, fixes numpy#6085, fixes numpy#3351, fixes numpy#6085, fixes numpy#6314,
fixes numpy#2346, fixes numpy#7058, fixes numpy#3641, fixes numpy#5994, fixes numpy#7262,
fixes numpy#7493
eric-wieser added a commit to eric-wieser/numpy that referenced this issue Sep 25, 2017
@eric-wieser
MAINT: Remove hack from numpygh-7659 for numpygh-7493
366b14c 
Working with 0d arrays is enough here
@eric-wieser eric-wieser mentioned this issue Sep 25, 2017
Merged
theodoregoetz pushed a commit to theodoregoetz/numpy that referenced this issue Oct 23, 2017
@ahaldane @theodoregoetz
MAINT: improve struct assignment, allow multi fields views
ae582a8 
This commit attempts to make structure assignment more consistent, and
then changes multi-field indices to return a view instead of a copy.

Assignment between structures now works "by field position" rather than
"by field name".

Fixes numpy#2353, fixes numpy#6085, fixes numpy#3351, fixes numpy#6085, fixes numpy#6314,
fixes numpy#2346, fixes numpy#7058, fixes numpy#3641, fixes numpy#5994, fixes numpy#7262,
fixes numpy#7493
theodoregoetz pushed a commit to theodoregoetz/numpy that referenced this issue Oct 23, 2017
@eric-wieser @theodoregoetz
MAINT: Remove hack from numpygh-7659 for numpygh-7493
ce4a720 
Working with 0d arrays is enough here
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug component: numpy.ma masked arrays
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

BUG: Temporary fix for str(mvoid) for object field types ahaldane/numpy
3 participants
@charris @kawing-chiu @ahaldane