Skip to content

Bug: MaskedArray object comparisons fail if mask is set due to bad fill value #14727

Closed
@mhvk

Description

@mhvk

A very puzzling issue in astropy - astropy/astropy#9374 - uncovered another bug in MaskedArray. It is not extremely easy to hit, but the following shows the problem

import numpy as np
ma = np.ma.MaskedArray([1, 3], dtype=object)
ma > 2
# masked_array(data=[False,  True],
#              mask=False,
#        fill_value=True)
ma._fill_value
# <None>
ma.fill_value
# '?'
ma._fill_value
# '?'
ma > 2
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/usr/lib/python3/dist-packages/numpy/ma/core.py in _check_fill_value(fill_value, ndtype)
    473             try:
--> 474                 fill_value = np.array(fill_value, copy=False, dtype=ndtype)
    475             except (OverflowError, ValueError):

ValueError: invalid literal for int() with base 10: '?'

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-6-c84cf3819565> in <module>()
----> 1 ma > 2

/usr/lib/python3/dist-packages/numpy/ma/core.py in __array_finalize__(self, obj)
   3017         # Finalize the fill_value
   3018         if self._fill_value is not None:
-> 3019             self._fill_value = _check_fill_value(self._fill_value, self.dtype)
   3020         elif self.dtype.names is not None:
   3021             # Finalize the default fill_value for structured arrays

/usr/lib/python3/dist-packages/numpy/ma/core.py in _check_fill_value(fill_value, ndtype)
    478                 # that the passed fill_value is not compatible with the ndtype.
    479                 err_msg = "Cannot convert fill_value %s to dtype %s"
--> 480                 raise TypeError(err_msg % (fill_value, ndtype))
    481     return np.array(fill_value)
    482 

TypeError: Cannot convert fill_value ? to dtype bool

The problem seems to be that code in MaskedArray counts on fill_value.dtype being the sort-of similar to that of the array, yet here it is not. Note that the following does work:

ma._fill_value = np.array('?', object)
ma > 2
# masked_array(data=[False,  True],
#              mask=False,
#        fill_value='?')
# NOTE: fill_value is wrong...
ma
# masked_array(data=[1, 3],
#              mask=False,
#        fill_value='?',
#             dtype=object)
# NOTE: repr is odd.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions