ENH: allow image to interpolate post RGBA #18782

jklymak · 2020-10-21T02:51:02Z

PR Summary

This implements a new boolean kwarg interp_postrgba to imshow that implements the interpolation after the data has been normed and mapped to RGBA.

Motivation

In the depths of time, we interpolated images in RGBA space, but this was deemed unacceptable in that it produces image pixels that are not in the colormap. So the interpolation was moved to be before the norm and colormapping. That has proven troublesome with respect to over/under data, but generally seems to work.

However, there are many cases, particularly when trying to antialias, that we really want to average in RGBA space, and allow colors that are out of the colormap because they visually merge adjacent pixels.

Proposal:

A kwarg, interp_space that allows the interpolation to be carried out before or after the RGBA mapping has been made. The way this is implemented now, it is just a toggle - i.e. you cannot interpolate both before and after the RGBA mapping. That is not impossible, but probably not typically needed or desired, and would make the code considerably messier. As it is, its quite a straightforward change.

Note the change could be made a level higher in _axes.imshow, however, this implementation allows us to skip all the mapping jiggery that we do for data space.

Result

import matplotlib.pyplot as plt
import numpy as np
import matplotlib.colors as mcolors
import matplotlib.cm as cm
import copy

fig, axs = plt.subplots(2, 2, figsize=(5.5, 5.5), sharex=False, sharey=False,
                        constrained_layout=True)
N = 250
aa = np.ones((N, N))
aa[::2, :] = -1

x = np.arange(N) / N - 0.5
y = np.arange(N) / N - 0.5

X, Y = np.meshgrid(x, y)
R = np.sqrt(X**2 + Y**2)
f0 = 10
k = 75
a = np.sin(np.pi * 2 * (f0 * R + k * R**2 / 2))

a[:int(N/2),:][R[:int(N/2),:]<0.4] = -1
a[:int(N/2),:][R[:int(N/2),:]<0.3] = 1
aa[:, int(N/2):] = a[:, int(N/2):]

aa[20:50, 20:50] = np.NaN
aa[70:90, 70:90] = 1e6
aa[70:90, 20:30] = -1e6
aa[ 70:90, 195:215,] = 1e6
aa[20:30, 195:215] = -1e6

cmap = copy.copy(cm.RdBu_r)
cmap.set_over('yellow')
cmap.set_under('cyan')

axs = axs.flatten()
axs[0].imshow(aa, interpolation='nearest', cmap=cmap, vmin=-1.2, vmax=1.2)
axs[0].set_title('Zoomed')
axs[0].set_xlim([N/2-25, N/2+25])
axs[0].set_ylim([N/2+50, N/2-10])

axs[1].imshow(aa, interpolation='nearest', cmap=cmap, vmin=-1.2, vmax=1.2)
axs[1].set_title('No antialiasing')

axs[2].imshow(aa, interpolation='antialiased', cmap=cmap, vmin=-1.2, vmax=1.2)
axs[2].set_title('Data antialiasing')
pc = axs[3].imshow(aa, interpolation='antialiased', interp_space='rgba',
                   cmap=cmap, vmin=-1.2, vmax=1.2 )
axs[3].set_title('RGBA antialiasing')

for ax in axs:
    ax.set_xticklabels([])
    ax.set_yticklabels([])
plt.show()

for dpi in [50, 100, 200, 400, 800]:
    fig.savefig(f'res{dpi}.png', dpi=dpi)

400 dpi

At 400 dpi the images are not downsampled, so all three versions are the same

200 dpi

Notice how at 100 dpi the alternating blue/red in the data interpolation go to white (the average of -1 and 1), with some obvious residual aliasing. The RBGA interpolation goes to purples (red+blue = purple).

100 dpi

50 dpi

Note that pixel peeping at the "over" and "under" you can definitely see pixels that leak out of the colormap (i.e. cyan+purple, or yellow + blue).

PR Checklist

Has pytest style unit tests (and pytest passes).
Is Flake 8 compliant (run flake8 on changed files to check).
New features are documented, with examples if plot related.
Documentation is sphinx and numpydoc compliant (the docs should build without error).
Conforms to Matplotlib style conventions (install flake8-docstrings and pydocstyle<4 and run flake8 --docstring-convention=all).
New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).

jklymak · 2020-10-21T18:17:06Z

This just needs tests and improved docs. But the idea is complete and could be discussed. @tacaswell @brunobeltran @efiring @dopplershift @anntzer ?

jklymak · 2020-12-18T15:16:26Z

ping @tacaswell - if you had time you said you'd weigh in on this one. I actually think this is a pretty simple addition, even if it will be a bit tricky to explain...

jklymak · 2021-03-31T16:27:59Z

#19838 is another nice example where data-interpolation on down-sampling is very undesirable.

jklymak · 2021-07-09T01:42:41Z

For Comment: this kwarg is currently interp_postrgba=True/False. I wonder if it should just be antialiased=True/False and just apply our favourite anti-aliasing filter. I'm also not clear if it should be independent of interpolation...

anntzer · 2021-07-10T08:53:37Z

IIUC #19748 (comment) essentially suggests that this should rather be something like interp_space={"raw"/"orig", "normed", "cmapped"}?

jklymak · 2021-07-19T18:03:03Z

OK, so changed to interp_space = "data"/"rgba". That leaves adding interp_sapce="normed" for future work. I'm not sure how we want to do that exactly, and I don't think anything in this PR stops that from working.

anntzer · 2021-07-19T18:21:41Z

lib/matplotlib/axes/_axes.py

@@ -5380,6 +5381,12 @@ def imshow(self, X, cmap=None, norm=None, aspect=None,
            which can be set by *filterrad*. Additionally, the antigrain image
            resize filter is controlled by the parameter *filternorm*.

+        interp_space : str, default: None


str -> {'data', 'rgba'}, default: 'data'

anntzer · 2021-07-19T18:21:50Z

lib/matplotlib/axes/_axes.py

+        interp_space : str, default: None
+            Supported values are 'data' and 'rgba'.  If 'data' interpolation
+            is carried out on the data provided by the user.  If 'rgba', the
+            interpolation is carried out afte the colormapping has been


after (typo)

anntzer · 2021-07-19T18:22:56Z

lib/matplotlib/axes/_axes.py

-               interpolation=None, alpha=None, vmin=None, vmax=None,
-               origin=None, extent=None, *, filternorm=True, filterrad=4.0,
-               resample=None, url=None, **kwargs):
+               interpolation=None, interp_space=None, alpha=None,


I don't think anyone is passing alpha positionally, so I guess the API break is safe, but in that case you may as well make all kwargs starting from interp_space keyword-only (as you effectively broke backcompat on passing alpha positionally).

OK, not intentionally. How far down the stack do we want to break it, if we had our druthers? I'd probably put the * after norm and before aspect?

Seems about right to me. OTOH you can also just stick interp_space in the kwonly section for now and put a @_api.make_keyword_only("3.5", "aspect") and fix the order in 3.7 to strictly follow the policy. No strong opinion there.

I deprecated most of them! (ahem, maybe we don't want this, but I really doubt too many people are using positional arguments for all of these)

you only need to do it for the first one; the following ones implicitly become keyword-only as well (because you can't pass them positionally anymore without passing the first one positionally) :)

ooops, duh!

If I do this, I get an error on import

Traceback (most recent call last): File "/Users/jklymak/matplotlib/examples/images_contours_and_fields/image_antialiasing.py", line 20, in <module> import matplotlib.pyplot as plt File "/Users/jklymak/matplotlib/lib/matplotlib/pyplot.py", line 2875, in <module> def imshow( File "/Users/jklymak/matplotlib/lib/matplotlib/pyplot.py", line 101, in _copy_docstring_and_deprecators func = decorator(func) File "/Users/jklymak/matplotlib/lib/matplotlib/_api/deprecation.py", line 417, in make_keyword_only assert (name in signature.parameters AssertionError: Matplotlib internal error: 'aspect' must be a positional-or-keyword parameter for imshow()

Oops, I guess (I'm almost sure, actually) that make_keyword_only currently doesn't work with functions that have pyplot wrappers, because make_keyword_only fakes the displayed function signature to pretend that the switch to kwonly already occurred, but the pyplot wrapper should be generated using the old (non-kwonly) signature, but that's not the case right now.
It's going to be slightly annoying to fix but probably just needs some wrangling, can you open a separate issue to track this? In any case I don't think this is required for this PR.

anntzer · 2021-07-19T18:23:56Z

lib/matplotlib/image.py

+        Parameters
+        ----------
+        s : str or None
+            str should be one on "data", "normed", or "rgba"


not "normed", for now.
ditto for the parameter description (as above)

anntzer · 2021-07-19T18:24:08Z

lib/matplotlib/image.py

+            s = "data"  # placeholder for maybe having rcParam
+        if s not in ["data", "rgba"]:
+            raise ValueError("interp_space must be one of 'data' "
+                             "or 'rgba'.")


_api.check_in_list.

anntzer · 2021-07-19T18:24:28Z

lib/matplotlib/image.py

+        Supported values are 'data' and 'rgba'.  If 'data' interpolation
+        is carried out on the data provided by the user.  If 'rgba', the
+        interpolation is carried out afte the colormapping has been
+        applied  (visual interpolation).


one space too many before the last opening parenthesis.

anntzer · 2021-07-19T21:34:29Z

lgtm, modulo tests/docs. Thanks for doing this :)

jklymak · 2021-07-20T13:48:45Z

@tacaswell I'll try and pop this onto your review queue, at least for high-level comment

tacaswell

I would like a version of #18487 to go in with this.

I'm still not fully convinced that this is not wrong, but have accepted that my instinct is from a career of working with data from 2D detectors and this behavior is sensible in some domains.

Another way to look at this is that it is putting in an option to get back to the mpl <2.0 behavior.

tacaswell · 2021-07-21T03:07:00Z

lib/matplotlib/image.py

@@ -885,6 +911,7 @@ def __init__(self, ax,
                 cmap=None,
                 norm=None,
                 interpolation=None,
+                 interp_space=None,


This should go at the end (kwarg only preferably), or we need a deprecation note.

tacaswell · 2021-07-21T03:07:05Z

examples/images_contours_and_fields/image_antialiasing.py

+#
+# Conversely, when the anti-aliasing occurs in 'rgba' space, the red and
+# blue are combined visually to make purple.  This behaviour is more like a
+# typical image processing package.


Suggested change

# typical image processing package.

# typical image processing package but may produce colors in output which are not present

# in the colormap confounding interpretation of the image.

Changed to

#... This behaviour is more like a # typical image processing package, but note that purple is not in the # original colormap, so it is no longer possible to invert individual # pixels.

The point here is that if you have adequate resolution, you should not be inverting individual pixels.

tacaswell · 2021-07-21T03:08:54Z

examples/images_contours_and_fields/image_antialiasing.py

+subsampling the smoothed data.  In Matplotlib, we can do that
+smoothing before mapping the data to colors, or we can do the smoothing
+on the RGB(A) data in the final image.  The difference between these is
+shown below, and controlled with the *interp_space* keyword argument.


I am not a huge fan of this name, I think of this more as changing the order of the pipeline rather than changing the space of the interpolation. I agree it does change the space, but the re-ordering (from resample -> norm -> colormap to norm -> colormap -> resample) is the key difference.

I'm not wed to the name, which was suggested by @anntzer, though I think its pretty good: interp_stage? interp_step?

jklymak · 2021-07-21T04:22:51Z

I would like a version of #18487 to go in with this.

I don't have any problem with that. Are you suggesting it should be part of this PR?

discussed on call

jklymak · 2021-07-24T15:14:17Z

Name of the kwarg has been changed to interpolation_stage.

doc/users/next_whats_new/image_interpolation_rgbastage.rst

examples/images_contours_and_fields/image_antialiasing.py

lib/matplotlib/axes/_axes.py

lib/matplotlib/image.py

Co-authored-by: Elliott Sales de Andrade <quantum.analyst@gmail.com>

jklymak added the topic: images label Oct 21, 2020

jklymak marked this pull request as draft October 21, 2020 02:53

jklymak linked an issue Oct 21, 2020 that may be closed by this pull request

imshow padding around NaN values #18735

Closed

jklymak mentioned this pull request Oct 21, 2020

imshow padding around NaN values #18735

Closed

jklymak mentioned this pull request Jan 24, 2021

imshow with very large arrays not working as expected #19276

Closed

jklymak mentioned this pull request Feb 1, 2021

Doc update antialiasing #18749

Closed

7 tasks

jklymak force-pushed the enh-image-postrgba-interpolate branch from 5017129 to f50ab9a Compare March 21, 2021 16:02

jklymak mentioned this pull request Mar 21, 2021

Incorrect & inconsistent coloring in .imshow() with LogNorm #19748

Closed

jklymak added this to the v3.5.0 milestone Mar 21, 2021

jklymak linked an issue Mar 21, 2021 that may be closed by this pull request

Incorrect & inconsistent coloring in .imshow() with LogNorm #19748

Closed

jklymak requested a review from tacaswell March 21, 2021 16:04

jklymak mentioned this pull request Mar 31, 2021

Unexpected behaviour of imshow default interpolation #19838

Closed

jklymak linked an issue Mar 31, 2021 that may be closed by this pull request

Unexpected behaviour of imshow default interpolation #19838

Closed

This was referenced Apr 7, 2021

[bugfix] Fix log plots when using matplotilb 3.3.x if there are zeros in the plot yt-project/yt#3161

Merged

[bug] errors in using log normalization with large data ranges yt-project/yt#3170

Closed

jklymak mentioned this pull request Jun 23, 2021

Documentation on Interpolations for imshow and Image antialiasing #20494

Closed

jklymak linked an issue Jun 23, 2021 that may be closed by this pull request

Documentation on Interpolations for imshow and Image antialiasing #20494

Closed

jklymak force-pushed the enh-image-postrgba-interpolate branch from f50ab9a to 0711ed3 Compare July 8, 2021 23:53

jklymak added the status: needs comment/discussion needs consensus on next step label Jul 9, 2021

jklymak force-pushed the enh-image-postrgba-interpolate branch from 40bd8a9 to 5560fb0 Compare July 19, 2021 18:10

anntzer reviewed Jul 19, 2021

View reviewed changes

jklymak mentioned this pull request Jul 19, 2021

make_keyword_only doesn't work for pyplot-wrapped methods #20684

Closed

jklymak force-pushed the enh-image-postrgba-interpolate branch from 0c855aa to a90a206 Compare July 19, 2021 23:47

jklymak marked this pull request as ready for review July 20, 2021 04:08

jklymak added status: needs review and removed status: needs comment/discussion needs consensus on next step labels Jul 20, 2021

jklymak force-pushed the enh-image-postrgba-interpolate branch from 197516b to ef3f515 Compare July 20, 2021 20:31

tacaswell previously requested changes Jul 21, 2021

View reviewed changes

jklymak force-pushed the enh-image-postrgba-interpolate branch from 3f07bff to 548a2df Compare July 24, 2021 04:53

QuLogic reviewed Aug 12, 2021

View reviewed changes

ENH: allow image to interpolate post RGBA

c460dbc

Co-authored-by: Elliott Sales de Andrade <quantum.analyst@gmail.com>

jklymak force-pushed the enh-image-postrgba-interpolate branch from 8793e03 to c460dbc Compare August 12, 2021 13:58

tacaswell approved these changes Aug 13, 2021

View reviewed changes

QuLogic removed the status: needs review label Aug 13, 2021

QuLogic approved these changes Aug 13, 2021

View reviewed changes

QuLogic merged commit bd42020 into matplotlib:master Aug 13, 2021

anntzer mentioned this pull request Feb 1, 2024

Add an rcparam for image.interpolation_stage. #27730

Merged

5 tasks

	# typical image processing package.
	# typical image processing package but may produce colors in output which are not present
	# in the colormap confounding interpretation of the image.

Uh oh!

ENH: allow image to interpolate post RGBA #18782

ENH: allow image to interpolate post RGBA #18782

Uh oh!

Conversation

jklymak commented Oct 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Motivation

Proposal:

Result

400 dpi

200 dpi

100 dpi

50 dpi

PR Checklist

Uh oh!

jklymak commented Oct 21, 2020

Uh oh!

jklymak commented Dec 18, 2020

Uh oh!

jklymak commented Mar 31, 2021

Uh oh!

jklymak commented Jul 9, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anntzer commented Jul 10, 2021

Uh oh!

jklymak commented Jul 19, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anntzer Jul 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anntzer commented Jul 19, 2021

Uh oh!

jklymak commented Jul 20, 2021

Uh oh!

tacaswell left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jklymak commented Jul 21, 2021

Uh oh!

jklymak commented Jul 24, 2021

Uh oh!

Uh oh!

jklymak commented Oct 21, 2020 •

edited

Loading

jklymak commented Jul 9, 2021 •

edited

Loading

anntzer Jul 19, 2021 •

edited

Loading