Skip to content

Indexing multidimensional structured masked array fails to translate fill_value correctly, leading to broadcasting errors ValueError when calling .filled() #6723

Closed
@gerritholl

Description

@gerritholl

When using a structured masked array where one of the structure elements has a multidimensional dtype, indexing does not correctly set the fill_value attribute for the new masked array. This leads to a fill_value with a shape that is incompatible with .data or .mask, which inevitably leads to broadcasting problems when fill_value is used, such as when calling filled():

In [332]: A = ma.masked_array(data=[([0,1,2],), ([3,4,5],)], mask=[([True, False, False],), ([False, True, False],)], dtype=[("A", ">i2", (3,))])

In [339]: A["A"][:, 0].filled()
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-339-ddde509d73bf> in <module>()
----> 1 A["A"][:, 0].filled()

/home/users/gholl/venv/stable-3.5/lib/python3.5/site-packages/numpy/ma/core.py in filled(self, fill_value)
   3573             result = self._data.copy('K')
   3574             try:
-> 3575                 np.copyto(result, fill_value, where=m)
   3576             except (TypeError, AttributeError):
   3577                 fill_value = narray(fill_value, dtype=object)

ValueError: could not broadcast input array from shape (3) into shape (2)

In [341]: A.shape, A.fill_value.shape, A["A"].shape, A["A"].fill_value.shape, A["A"][:, 0].shape, A["A"][:, 0].fill_value.shape
Out[341]: ((2,), (), (2, 3), (3,), (2,), (3,))

Indeed, the fill value doesn't change upon indexing the masked array — only its dtype does:

In [351]: A.fill_value, A["A"].fill_value, A["A"][:, 0].fill_value
Out[351]: 
(([16959, 16959, 16959],),
 array([16959, 16959, 16959], dtype=int16),
 array([16959, 16959, 16959], dtype=int16))

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions