Skip to content

[Bug]: regression with LineCollection + TwoSlopeNorm colorbar in matplotlib 3.5.0b1 #20993

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
neutrinoceros opened this issue Sep 4, 2021 · 18 comments

Comments

@neutrinoceros
Copy link
Contributor

Bug summary

Some image comparison tests in a project of mine are failing when running matplotlib 3.50b1

Code for reproduction

import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
from matplotlib.colors import TwoSlopeNorm
import numpy as np

x = np.linspace(0, 10, 100)
y = np.sin(x)
# generate some data within some interval that doesn't have 0 its central value
z = y + 0.5

points = np.array([x, y]).T.reshape(-1, 1, 2)
segments = np.concatenate([points[:-1], points[1:]], axis=1)

# adapted from
# https://matplotlib.org/stable/gallery/lines_bars_and_markers/multicolored_line.html
lc = LineCollection(segments, norm=TwoSlopeNorm(0), cmap="RdYlBu")
lc.set_array(z)

fig, ax = plt.subplots()

# this line is just to compensate for the fact that the limits are not
# automatically handled when the plot just contains a single LineCollection artist
ax.set(xlim=(0, 10), ylim=(-1, 1))

line = ax.add_collection(lc)
fig.colorbar(line)

Actual outcome

linecollection_mpl_v3 5 0b1

Expected outcome

This is the results with matplotlib 3.4.3
linecollection_mpl_v3 4 3

Operating system

OS/X

Matplotlib Version

3.5.0b1

Matplotlib Backend

MacOSX

Python version

3.9.6

Jupyter version

N/A

Other libraries

No response

Installation

pip

Conda channel

No response

@neutrinoceros
Copy link
Contributor Author

I should add that this seems specific to LineCollection. Now if I run a similar test for a contour plot

import matplotlib.pyplot as plt
from matplotlib.colors import TwoSlopeNorm
import numpy as np

x, y = np.mgrid[1:100, 1:100]
z = np.random.random_sample(x.shape) - 0.75

fig, ax = plt.subplots()
im = ax.contourf(x, y, z, levels=50, cmap="RdYlBu", norm=TwoSlopeNorm(0))
cbar = fig.colorbar(im)

I get consistent results in matplotlib 3.4.3 and 3.5.0b1
contourf_mpl_v3 5 0b1
This image was obtained with 3.5.0b1 and the behaviour is consistent with what the colorbar looks like for a LineCollection object in matplotlib 3.4.3 shown in the OP

@jklymak
Copy link
Member

jklymak commented Sep 4, 2021

The 3.5 example above is an expected change and correct in my opinion.

@neutrinoceros
Copy link
Contributor Author

In any case the behavior seems inconsistent between LineCollection and contour plots. Actually both outcomes seem reasonable to me as a user, and I would not mind too much if the change was actually embraced in the next release, I'm mostly worried that having such inconsistencies will make maintenance much harder down the road.

@jklymak
Copy link
Member

jklymak commented Sep 5, 2021

There are many inconsistencies between contour colorbars compared to other scalar mapable colorbars. For instance:

import matplotlib.pyplot as plt
from matplotlib.colors import TwoSlopeNorm, LogNorm
import numpy as np

x, y = np.mgrid[1:100, 1:100]
z = np.abs(np.random.randn(*x.shape))

fig, ax = plt.subplots(1, 2, constrained_layout=True)
im = ax[0].contourf(x, y, z, levels=50, cmap="RdYlBu", norm=LogNorm(vmin=0.01, vmax=10))
cbar = fig.colorbar(im, ax=ax[0])

im = ax[1].pcolormesh(x, y, z, cmap="RdYlBu", norm=LogNorm(vmin=0.01, vmax=10))
cbar = fig.colorbar(im, ax=ax[1])
plt.show()

yields:

Out

While I think that can/should be improved to be consistent with other scalar mappables, I don't think there is a regression, because I think this has been this way for a long time.

@neutrinoceros
Copy link
Contributor Author

Feel free to change the title of the ticket if it helps triaging it.

@jklymak
Copy link
Member

jklymak commented Sep 5, 2021

Sorry, that example was a bit misleading because of #19856

However, a better example is

Ex2

import matplotlib.pyplot as plt
from matplotlib.colors import TwoSlopeNorm, LogNorm
import numpy as np

x, y = np.mgrid[1:100, 1:100]
z = np.abs(np.random.randn(*x.shape))

fig, ax = plt.subplots(2, 2, constrained_layout=True)
im = ax[0, 0].contourf(x, y, z, levels=np.linspace(0.1, 10, 20), cmap="RdYlBu", norm=LogNorm(vmin=0.1, vmax=10))
cbar = fig.colorbar(im, ax=ax[0, 0])
ax[0, 0].set_title('linear spaced contours')

im = ax[0, 1].contourf(x, y, z, levels=np.logspace(-1, 1, 20), cmap="RdYlBu", norm=LogNorm(vmin=0.1, vmax=10))
cbar = fig.colorbar(im, ax=ax[0, 1])
ax[0, 1].set_title('logarithmically spaced contours')

im = ax[1, 0].pcolormesh(x, y, z, cmap="RdYlBu", norm=LogNorm(vmin=0.1, vmax=10))
ax[1, 0].set_title('pcolormesh')
cbar = fig.colorbar(im, ax=ax[1, 0])
fig.savefig('/Users/jklymak/downloads/Ex2.png', dpi=100)
plt.show()

So, what is happening here? First note that for all three examples the colormapping is the same. 1.0 is the middle of the colormap.

The colorbar is plotting the contours equally spaced on the axes, such that each contour level gets the same amount of space on the colorbar. When you specify the contours linearly, there are far more of them above 1.0 than below, and that part of the axes is emphasized. The axes are formatted with a default formatter.

Its possible we could toggle this to have the colorbar follow the norm, but that would be a feature request.

@neutrinoceros
Copy link
Contributor Author

Your explanation with contour plots makes a lot of sense. However, it seems to me that it actually reinforces my point

there are far more of them above 1.0 than below, and that part of the axes is emphasized.

So in my OP, there are more levels above 0 than bellow, so I'm expecting the colorbar to emphasise this, which is exactly what matplotlib 3.4.3 does. I do not understand how the 3.5.0b1 version is doing it better.
Perhaps more importantly, it doesn't seem like there's an easy way for me to reproduce my figures as I know them if I eventually migrate to 3.5. Is there ?

@jklymak
Copy link
Member

jklymak commented Sep 6, 2021

The two-slope norm puts the same number of colours above and below the midpoint, and the new colorbar in 3.5 properly reflects that.

If you want the old behaviour, I believe you can just change the colorbar scale to be linear. (Away from computer so I'd have to check the details)

@jklymak
Copy link
Member

jklymak commented Sep 6, 2021

As below, you can just set the colorbar yscale:

import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
from matplotlib.colors import TwoSlopeNorm
import numpy as np

x = np.linspace(0, 10, 100)
y = np.sin(x)
# generate some data within some interval that doesn't have 0 its central value
z = y + 0.5

points = np.array([x, y]).T.reshape(-1, 1, 2)
segments = np.concatenate([points[:-1], points[1:]], axis=1)

fig, axs = plt.subplots(1, 2, constrained_layout=True)

for ind in range(2):
    ax = axs[ind]
    # this line is just to compensate for the fact that the limits are not
    # automatically handled when the plot just contains a single LineCollection artist
    ax.set(xlim=(0, 10), ylim=(-1, 1))
    lc = LineCollection(segments, norm=TwoSlopeNorm(0), cmap="RdYlBu")
    lc.set_array(z)

    line = ax.add_collection(lc)
    cbar = fig.colorbar(line, ax=ax)
    if ind == 1:
        cbar.ax.set_yscale('linear')
        ax.set_title('Linear colorbar')
    else:
        ax.set_title('Default colorbar')
fig.savefig('/Users/jklymak/downloads/Ex3.png', dpi=100)
plt.show()

I wanted to add cb.set_scale, but that may still be future discussion.

Ex3

@neutrinoceros
Copy link
Contributor Author

Great ! I'll use that then.

I wanted to add cb.set_scale, but that may still be future discussion.

yeah well, Rome wasn't built in a day. I deeply appreciate the gigantic effort you've been undertaking, and the time you've been pouring in answering my annoying questions and reports :)

I think this can be closed now, thank you so much for you help !

@jklymak
Copy link
Member

jklymak commented Sep 6, 2021

Well norms are definitely confusing. I think the confusing thing here is when the norm gets applied, before or after digitization by contour levels or the finite colormap. One of the main points of #20054 was to make norms other than log norm behave like lognorm. OTOH, if the release notes aren't clear enough about that, we should try and fix them before the final release.

@neutrinoceros
Copy link
Contributor Author

TBH it didn't occur to me that this change could be desired when I reported, so I didn't lookup the release notes.

@jklymak
Copy link
Member

jklymak commented Sep 6, 2021

Again, consider the logarithmic case; almost certainly you do not want a linear axes in that case. For the two-slope norm, presumably if you want a different magnification on either side of zero you also want different resolution in the scale.

@neutrinoceros
Copy link
Contributor Author

In the future, probably. Right now I'm just trying to maintain existing scripts for figures that are part of a scientific publication, so reproducibility is what I care most about here :)

@jklymak
Copy link
Member

jklymak commented Sep 6, 2021

You could package your environment somehow rather than try and make it work on future versions of everything.

@neutrinoceros
Copy link
Contributor Author

True. I actually wasn't trying, just accidentally ended up testing with a beta version

@tacaswell
Copy link
Member

It is yet another API change, but we should also think about making the Locators on colorbars default to putting a tick on the top/bottom of the range.

@jklymak
Copy link
Member

jklymak commented Sep 7, 2021

@tacaswell we don't delimit the top and bottom for normal axes. Is it desirable for colorbars for some reason?

In the case of two-slope norm, it is currently using the linear locator, but it should probably have a custom locator that has a tick at the midpoint, and ticks out differently above and below the midpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants