Skip to content

[Bug]: imshow allows RGB(A) images with np.nan values to pass #27301

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
shaperilio opened this issue Nov 9, 2023 · 7 comments · Fixed by #27848
Closed

[Bug]: imshow allows RGB(A) images with np.nan values to pass #27301

shaperilio opened this issue Nov 9, 2023 · 7 comments · Fixed by #27848
Milestone
v3.9.0

Comments

@shaperilio
Copy link

shaperilio commented Nov 9, 2023 •
edited
Loading

Bug summary

Using imshow to visualize a 2x2x1 array with nans scattered throughout works as expected. With the default "bad" color of black with alpha = 0, you'll see an axis' facecolor where the nans are.

With a 2x2x3 array, the nans are always black.

With a 2x2x4 array, and the alpha manually set to 0 where the nans are, you get the correct behavior (i.e. you can see the axis where the nans are).

Code for reproduction

import numpy
from matplotlib import pyplot

img = numpy.ones((2, 2))  # single channel 2x2
img_nan = img.copy()
img_nan[0, 0] = float('nan')

img3 = numpy.ones((2, 2, 3))  # RGB 2x2
img3_nan = img3.copy()
img3_nan[0, 0, :] = float('nan')

img4 = numpy.ones((2, 2, 4))  # RGBA 2x2
img4_nan = img4.copy()
img4_nan[0, 0, :] = float('nan')
img4_nan[0, 0, 3] = 0  # alpha


def imshow(a):
    pyplot.figure()
    im = pyplot.imshow(a)
    # im.get_cmap().set_bad(color='y', alpha=1) # only works for single channel images
    im.get_cmap().set_bad(alpha=0)
    pyplot.gca().set_facecolor('y')


imshow(img_nan)   # nan shows up as 'y'
imshow(img3_nan)  # nan shows up as black
imshow(img4_nan)  # nan shows up as 'y'


pyplot.show()

Actual outcome

image
image
image

Expected outcome

Figures 2 and 3 above should look identical.

Additional information

I traced through imshow and cannot see any difference between the single- and 3-channel image; the mask is there and it's correct in both cases. So I wonder if this is a backend issue?

Operating system

Windows

Matplotlib Version

3.7.3

Matplotlib Backend

QtAgg

Python version

3.8.10

Jupyter version

No response

Installation

pip
@WeatherGod
Copy link
Member

WeatherGod commented Nov 9, 2023 via email

@shaperilio
Copy link
Author

There's an additional thing here.

Moving the mouse pointer over the image:

  1. In the 1- and 3-channel case, the area where the nan is shows []
  2. in the 4-channel case, it shows [0]

@shaperilio
Copy link
Author

colormaps aren't used for 3 or 4 channel imshow()'s, so the "bad" is never applicable there. With the 4 channel one, having an alpha of zero just reveals the color underneath.

OK fine, colormaps are ignored for 3- and 4-channel images. I guess I'm arguing that masking out nan should still happen. It actually is, and then getting thrown out / ignored somewhere down the line.

@tacaswell
Copy link
Member

For float RGB(A) images I am not sure what I would expect np.nan to do. Empirically, it looks like it is getting converted to 0 by Agg (as ScalarMappable.to_rgba appears to pass it through just fine and does not trigger any of the warnings as the np.nan poisons the min/max computations and reports false for all comparisons).

My inclination is that the second two cases should be raising on invalid input (as they would if you passed 2 for one of the channels.

@tacaswell
Copy link
Member

tacaswell commented Nov 10, 2023 •
edited
Loading

and the reason the second RGB(A) case looks like it works is that the alpha channel is 0 so it does not matter what the RGB channels do.

@tacaswell tacaswell added this to the v3.9.0 milestone Nov 10, 2023
tacaswell added a commit to tacaswell/matplotlib that referenced this issue Nov 10, 2023
@tacaswell
FIX: also exclude np.nan in RGB(A) in color mapping
583188d 
closes matplotlib#27301
@tacaswell tacaswell changed the title [Bug]: set_bad doesn't work with imshow and 3-channel images [Bug]: imshow allows RGB(A) images with np.nan values to pass Nov 10, 2023
@tacaswell tacaswell mentioned this issue Nov 10, 2023
Closed
2 tasks
@shaperilio
Copy link
Author

@tacaswell, if I understand correctly, you're now going to raise an exception if an RGB image has nans in it?

This seems quite harsh, since single-channel images are allowed to have nans and even have a mechanism to control their display with set_bad.

My RGBA example is a workaround; in our application, I went that route because I want 1- and 3- channel images to behave the same with regards to nan.

@rcomer
Copy link
Member

rcomer commented Nov 10, 2023

I think I would expect points with nans to be treated the same as masked points. We have explicit handling that sets the alpha to zero if any of the channels are masked at the given point:

matplotlib/lib/matplotlib/cm.py

Lines 501 to 504 in c0e9ebc

# Account for any masked entries in the original array
# If any of R, G, B, or A are masked for an entry, we set alpha to 0
if np.ma.is_masked(x):
xx[np.any(np.ma.getmaskarray(x), axis=2), 3] = 0

@rcomer rcomer mentioned this issue Mar 3, 2024
Merged
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v3.9.0
4 participants
@tacaswell @WeatherGod @rcomer @shaperilio