Skip to content

Bug: MaskedArray object comparisons fail if mask is set due to bad fill value #14727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mhvk opened this issue Oct 16, 2019 · 2 comments
Closed

Comments

@mhvk
Copy link
Contributor

mhvk commented Oct 16, 2019

A very puzzling issue in astropy - astropy/astropy#9374 - uncovered another bug in MaskedArray. It is not extremely easy to hit, but the following shows the problem

import numpy as np
ma = np.ma.MaskedArray([1, 3], dtype=object)
ma > 2
# masked_array(data=[False,  True],
#              mask=False,
#        fill_value=True)
ma._fill_value
# <None>
ma.fill_value
# '?'
ma._fill_value
# '?'
ma > 2
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/usr/lib/python3/dist-packages/numpy/ma/core.py in _check_fill_value(fill_value, ndtype)
    473             try:
--> 474                 fill_value = np.array(fill_value, copy=False, dtype=ndtype)
    475             except (OverflowError, ValueError):

ValueError: invalid literal for int() with base 10: '?'

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-6-c84cf3819565> in <module>()
----> 1 ma > 2

/usr/lib/python3/dist-packages/numpy/ma/core.py in __array_finalize__(self, obj)
   3017         # Finalize the fill_value
   3018         if self._fill_value is not None:
-> 3019             self._fill_value = _check_fill_value(self._fill_value, self.dtype)
   3020         elif self.dtype.names is not None:
   3021             # Finalize the default fill_value for structured arrays

/usr/lib/python3/dist-packages/numpy/ma/core.py in _check_fill_value(fill_value, ndtype)
    478                 # that the passed fill_value is not compatible with the ndtype.
    479                 err_msg = "Cannot convert fill_value %s to dtype %s"
--> 480                 raise TypeError(err_msg % (fill_value, ndtype))
    481     return np.array(fill_value)
    482 

TypeError: Cannot convert fill_value ? to dtype bool

The problem seems to be that code in MaskedArray counts on fill_value.dtype being the sort-of similar to that of the array, yet here it is not. Note that the following does work:

ma._fill_value = np.array('?', object)
ma > 2
# masked_array(data=[False,  True],
#              mask=False,
#        fill_value='?')
# NOTE: fill_value is wrong...
ma
# masked_array(data=[1, 3],
#              mask=False,
#        fill_value='?',
#             dtype=object)
# NOTE: repr is odd.
@seberg
Copy link
Member

seberg commented Jun 21, 2022

Would be mostly solved by gh-21812, but I note also here: Fill value propagation seems pretty broken to me. At least when different dtypes are involved (i.e. the result array has a different dtype from the input one).

@mhvk
Copy link
Contributor Author

mhvk commented Jan 11, 2024

Actual issue solved by #21812; there may be other problems with fill_value, but they better get their own issue...

@mhvk mhvk closed this as completed Jan 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants