Cannot get a view from a composite array with object data members #3253

mwtoews · 2013-04-17T07:29:58Z

With NumPy 1.7.1 on Win64 with Python 2.7 I see a new issue that was not a problem on a previous version (e.g., 1.6.2).

Consider a composite data type, where a view is obtained from:

import numpy as np
dat1 = np.zeros(3, [('A', 'i'), ('B', 'f')])
new1 = dat1[['B', 'A']]
print(repr(new1))
# array([(0.0, 0), (0.0, 0), (0.0, 0)], 
#      dtype=[('B', '<f4'), ('A', '<i4')])

cool (note: back in 1.4.1 although a subset could be made, the order was the same as the original).

However, if one of types used in the composite data type is '|O8', I can no longer obtain a view:

dat2 = np.zeros(3, [('A', 'i'), ('B', '|O8')])
new2 = dat2[['B', 'A']]

Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\numpy\core_internal.py", line 296, in _index_fields
view = ary.view(dtype=view_dtype)
TypeError: Cannot change data-type for object array.

imstochastic · 2013-11-21T01:10:38Z

FWIW, still the case on "1.7.2.dev-3b1a809".

I have the use case where I want to have two datatypes: some base type 'dt' and another type 'dtSuper' which is a fixed-size array of 'dt' elements. I'd like to cast a properly-sized array with dtype dt to a (1,) array with dtype 'dtSuper', like so:

dt = np.dtype([('a','O'),('b','f8')])
a = np.zeros((5,),dtype=dt)
dtSuper = np.dtype([('samples',dt,(5,))])
a.view(dtSuper)

Object types, I imagine, could be dangerous if we're allowed to re-interpret them as arbitrary data types, as that gives us direct access to memory addresses and suddenly we're programming in C, with enough rope to hang ourselves with :-). I'm speculating that this is the underlying reason why we are now seeing this error in 1.7.1.

However the use case I just described doesn't have this problem. It seems that it would make sense to special-case the code, or else (to be more general) somehow check to ensure that every object pointer in the old dtype maps to an object pointer in the new dtype.

charris · 2014-02-22T20:52:10Z

Still open in 1.9-devel. Note that is error occurs in _internal.py, so might not be too difficult to fix.

charris · 2014-02-22T20:53:33Z

So labelled "easy fix" until proved otherwise ;)

Previously views of structured arrays containing objects were completely disabled. This commit adds more lenient check for whether an object-array view is allowed, and adds similar checks to getfield/setfield Fixes numpy#2346. Fixes numpy#3256. Fixes numpy#2599. Fixes numpy#3253. Fixes numpy#3286. Fixes numpy#5762.

charris added Defect labels Feb 22, 2014

ahaldane mentioned this issue Feb 9, 2015

ENH: structured datatype safety checks #5548

Merged

jaimefrio closed this as completed in e2cd6fa Jun 5, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot get a view from a composite array with object data members #3253

Cannot get a view from a composite array with object data members #3253

mwtoews commented Apr 17, 2013

imstochastic commented Nov 21, 2013

charris commented Feb 22, 2014

charris commented Feb 22, 2014

Cannot get a view from a composite array with object data members #3253

Cannot get a view from a composite array with object data members #3253

Comments

mwtoews commented Apr 17, 2013

imstochastic commented Nov 21, 2013

charris commented Feb 22, 2014

charris commented Feb 22, 2014