Add DivergingNorm (again, again, again) #12419

jklymak · 2018-10-06T04:11:04Z

PR Summary

Edit: see documentation here (and examples linked therein)

https://19203-1385122-gh.circle-artifacts.com/0/home/circleci/project/doc/build/html/api/_as_gen/matplotlib.colors.DivergingNorm.html#matplotlib.colors.DivergingNorm

This PR is re-invigorating #5054, which re-invigorated #4666, which re-invigorated #3858. Who knows, maybe it went back before that. Hat tip to @dopplershift, @OceanWolf , @jkseppan, @phobson who wrote this, not me. EDIT: forgot #5061....

It (re-) introduces DivergingNorm:

This seems like a pretty straightforward and helpful enhancement, so not sure why this is the fourth PR. OTOH, please note the todo-list is still active, this needs docs, whats new, and more thorough tests and I've not yet double checked the code. But throwing it out there in case any of the original authors want their PR back.

Mostly ready to go, but no doubt some edge cases...

    x = np.linspace(-2, 7)
    y = np.linspace(-1*np.pi, np.pi)
    X, Y = np.meshgrid(x, y)
    Z = x * np.sin(Y)**2

    fig, (ax1, ax2) = plt.subplots(ncols=2)
    cmap = plt.cm.coolwarm
    norm = mcolors.DivergingNorm(vmin=-2, vcenter=0, vmax=7)

    img1 = ax1.pcolormesh(Z, cmap=cmap, norm=None)
    cbar1 = fig.colorbar(img1, ax=ax1)

    img2 = ax2.pcolormesh(Z, cmap=cmap, norm=norm)
    cbar2 = fig.colorbar(img2, ax=ax2)

PR Checklist

Has Pytest style unit tests
Code is Flake 8 compliant
New features are documented, with examples if plot related
Documentation is sphinx and numpydoc compliant
Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

jklymak · 2018-10-06T20:08:14Z

Actually in the light of day I’m not going to champion this. If someone tried to publish in my field with negative anomalies on a different colour-to-data ratio than positive anomalies they would be in for a drubbing. Maybe there are applications that I don’t know about where this feature would be desirable and legitimate so feel free to reopen this or one of its predecessors. But, I don’t know the use case for encouraging this sort of data mapping by default.

ImportanceOfBeingErnest · 2018-10-06T20:31:52Z

The use of such Norm is clearly that you may work with one single colormap and adapt the norm to the data in use.

The following image is taken from this stackoverflow question where the idea is to define a "sealevel", such that the turquoise part of the colormap is always at 0 elevation.

The alternative to this approach would be to create a new colormap for each individual dataset, which is surely undesireable.

jklymak · 2018-10-06T20:41:44Z

Ahh. Thanks the sea level one is a great example where this is ok. I’ll change the examples and tests to that 😉

ImportanceOfBeingErnest · 2018-10-06T20:48:07Z

There is by the way a part about this in the docs: https://matplotlib.org/users/colormapnorms.html#custom-normalization-two-linear-ranges which tells that "This may appear soon as colors.OffsetNorm()"

and it has the warning and the use case mentionned:

As above, non-symmetric mapping of data to color is non-standard practice for quantitative data, and should only be used advisedly. A practical example is having an ocean/land colormap where the land and ocean data span different ranges.

jklymak · 2018-10-06T21:17:58Z

Ha ha, I wrote that 😉

ImportanceOfBeingErnest · 2018-10-06T22:22:01Z

In the current version that example has somehow become useless, as it suddenly uses data that is symmetric about 0 anyways. It also lost the warning. There is also an error in the BoundaryNorm section. I suppose that was edited in a hurry and would need to be refurbished anyways.

lib/matplotlib/colors.py

QuLogic · 2018-10-07T02:02:48Z

lib/matplotlib/tests/test_colors.py

 import numpy as np
 import pytest

-from numpy.testing import assert_array_equal, assert_array_almost_equal
+from numpy.testing import (assert_array_equal, assert_array_almost_equal,
+                           assert_equal)


I think we've generally moved away from using assert_equal. Especially because you're just comparing single values, just assert that the two things are equal.

lib/matplotlib/colors.py

QuLogic · 2018-10-07T02:07:04Z

lib/matplotlib/colors.py

+            vmax = np.ma.max(A)
+
+        if vmin > vmax:
+            raise ValueError('vmin must be less than vmax')


less than or equal (or change condition)?

lib/matplotlib/colors.py

phobson · 2018-10-08T16:08:20Z

lib/matplotlib/colors.py

-                raise ValueError('vmin, vcenter, and vmax must '
-                                 'increment monotonically for DivergingNorm '
+                raise ValueError('vmin, vcenter, and vmax must increment '
+                                 'monotonically for DivergingNorm '


minor point, but would it be more clear to say?

DivergingNorm requires that vmin <vcenter < vmax

jklymak · 2018-10-09T00:25:28Z

@pharshalp this now doesn't clip the values outside vmin, vmax unless clip is True...

pharshalp · 2018-10-09T02:14:53Z

@pharshalp this now doesn't clip the values outside vmin, vmax unless clip is True...

@jklymak Thanks! Just checked it using a small example and can confirm that it works!

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors as mcolors

x = np.linspace(-2, 7)
y = np.linspace(-1*np.pi, np.pi)
X, Y = np.meshgrid(x, y)
Z = x * np.sin(Y)**2

fig, (ax1, ax2) = plt.subplots(ncols=2)
cmap = plt.cm.coolwarm
cmap.set_over('orange')
cmap.set_under('steelblue')
norm = mcolors.DivergingNorm(vmin=-1, vcenter=0, vmax=6)

img1 = ax1.pcolormesh(Z, cmap=cmap, norm=None)
cbar1 = fig.colorbar(img1, ax=ax1, extend='neither')

img2 = ax2.pcolormesh(Z, cmap=cmap, norm=norm)
cbar2 = fig.colorbar(img2, ax=ax2, extend='both')

plt.show()

ImportanceOfBeingErnest · 2018-10-09T09:07:40Z

Any idea why the two new norms do not appear in the "classes" list in the docs?

jklymak · 2018-10-09T16:09:11Z

py 3.5 errors are due to slight differences in whether interpolation is under or over in numpy (i.e. does 1.0 interpolate into [0. 1.] as an over value or not - 3.6/3.7 say not, 3.5 says yes).

What is the plan for 3.1 wrt py 3.5? Discuss in #12358

jklymak · 2019-03-08T02:38:40Z

https://19203-1385122-gh.circle-artifacts.com/0/home/circleci/project/doc/build/html/api/_as_gen/matplotlib.colors.DivergingNorm.html#matplotlib.colors.DivergingNorm

examples/userdemo/colormap_normalizations_diverging.py

anntzer · 2019-03-13T18:54:45Z

lib/matplotlib/colorbar.py

+
+        bunique = b
+        yunique = self._y
+        if b[0] == b[1]:


I see how the previous code was doing linear interpolation by hand, but where do these "equal first and second point / equal last and nexttolast point" cases come from?

Fixed comment:

bunique = b yunique = self._y # trim extra b values at beginning and end if they are # not unique. These are here for extended colorbars, and are not # wanted for the interpolation.

I don't quite understand this. Souldn't you trim all but one duplicate values?
The current code does b = [0, 0, 0, 1, 2] --> bunique = [0, 0, 1, 2]. Is that intended?

That should never happen - if it does, just as well that an error trips. b are the boundaries, passed through the norm should be monotonic, except for, possibly, the first and last values if there are an "extend" either at max or min.

anntzer · 2019-03-13T18:58:05Z

examples/userdemo/colormap_normalizations_diverging.py

+    cmap=terrain_map,)
+ax.set_xlabel('Lon $[^o E]$')
+ax.set_ylabel('Lat $[^o N]$')
+ax.set_aspect(1 / np.cos(np.deg2rad(49)))


I guess that comes from Mercator? kind of obscure if that's not your field, but probably not a big deal.

Added explanation: This is just a local "projection" centered at 49 degrees taking into account how much closer the longitude lines are at that lattitude than at the equator and then assuming the lines of longitude are parallel. Not very sophisticated 😉

anntzer

Only minor nitpicks left, nothing blocking the merge.

jklymak · 2019-03-13T19:53:39Z

Thanks @anntzer. Reasonable nitpicks that I’ll fix today. EDIT: Fixed!

@Tillsen

Borrows heavily from @Tillsen's solution found on StackOverflow here: http://goo.gl/RPXMYB Used with his permission dicussesd on Github here: https://github.com/matplotlib/matplotlib/pull/3858` TST: add tests for DivergingNorm DOC: add to colors_api.rst DOC: add tutorial FIX: fix extend=both DOC: add new example

anntzer · 2019-03-14T12:49:22Z

lib/matplotlib/tests/test_colors.py

+    assert norm.scaled() is True
+
+
+def test_DivergingNorm_scaleout_center():


this and the one below could use pytest.mark.parametrize

anntzer · 2019-03-14T12:49:34Z

lib/matplotlib/tests/test_colors.py

+    assert norm.vmin == -5
+
+
+def test_DivergingNorm_Even():


and these two as well

anntzer · 2019-03-14T12:49:46Z

lib/matplotlib/tests/test_colors.py

+    assert_array_equal(norm(vals), expected)
+
+
+def test_DivergingNorm_VminEqualsVcenter():


and these fives

If someone wants to parametrize these post-facto they can. I am not a fan of parameterized tests....

jklymak · 2019-03-18T19:49:44Z

Thanks everyone, particularly @phobson for the original PR, and @anntzer for the thorough review

…419-on-v3.1.x Backport PR #12419 on branch v3.1.x (Add DivergingNorm (again, again, again))

jklymak force-pushed the add-asym-norm branch from 3341124 to 7fe02bc Compare October 6, 2018 04:15

This was referenced Oct 6, 2018

PiecewiseLinearNorm #5061

Closed

Diverging norm #5054

Closed

jklymak closed this Oct 6, 2018

jklymak reopened this Oct 6, 2018

jklymak mentioned this pull request Oct 6, 2018

ColorbarNorm Tutorial needs some work.... #12424

Closed

QuLogic reviewed Oct 7, 2018

View reviewed changes

jklymak force-pushed the add-asym-norm branch 8 times, most recently from db8c3de to 38cb3dc Compare October 7, 2018 05:37

phobson reviewed Oct 8, 2018

View reviewed changes

pharshalp mentioned this pull request Oct 8, 2018

[Wishlist/candidate for gallery?] Diverging colormap taking into account asymmetry of the data about vcenter #12449

Closed

jklymak force-pushed the add-asym-norm branch 2 times, most recently from 28c659b to 323d41f Compare October 9, 2018 00:25

jklymak mentioned this pull request Oct 9, 2018

Dropping support for Py3.5 and numpy 1.10 #12358

Closed

jklymak force-pushed the add-asym-norm branch 3 times, most recently from 9fe878e to 3d6c3c6 Compare March 8, 2019 01:47

jklymak added the status: needs review label Mar 8, 2019

jklymak requested review from anntzer and ImportanceOfBeingErnest March 13, 2019 18:18

anntzer reviewed Mar 13, 2019

View reviewed changes

examples/userdemo/colormap_normalizations_diverging.py Outdated Show resolved Hide resolved

anntzer reviewed Mar 13, 2019

View reviewed changes

examples/userdemo/colormap_normalizations_diverging.py Outdated Show resolved Hide resolved

anntzer reviewed Mar 13, 2019

View reviewed changes

examples/userdemo/colormap_normalizations_diverging.py Outdated Show resolved Hide resolved

anntzer reviewed Mar 13, 2019

View reviewed changes

anntzer approved these changes Mar 13, 2019

View reviewed changes

jklymak force-pushed the add-asym-norm branch 2 times, most recently from 1c81602 to 31785c2 Compare March 13, 2019 23:34

jklymak force-pushed the add-asym-norm branch from 31785c2 to c5b8d86 Compare March 14, 2019 02:30

anntzer reviewed Mar 14, 2019

View reviewed changes

lib/matplotlib/tests/test_colors.py

assert norm.vmin == -5

def test_DivergingNorm_Even():

Copy link

Contributor

anntzer Mar 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and these two as well

anntzer reviewed Mar 14, 2019

View reviewed changes

tacaswell merged commit 8cf381d into matplotlib:master Mar 18, 2019

meeseeksmachine pushed a commit to meeseeksmachine/matplotlib that referenced this pull request Mar 18, 2019

Backport PR matplotlib#12419: Add DivergingNorm (again, again, again)

0a77469

meeseeksmachine mentioned this pull request Mar 18, 2019

Backport PR #12419 on branch v3.1.x (Add DivergingNorm (again, again, again)) #13705

Merged

jklymak deleted the add-asym-norm branch March 18, 2019 19:49

timhoffm added a commit that referenced this pull request Mar 18, 2019

Merge pull request #13705 from meeseeksmachine/auto-backport-of-pr-12…

9cd8874

…419-on-v3.1.x Backport PR #12419 on branch v3.1.x (Add DivergingNorm (again, again, again))

QuLogic removed the status: needs review label May 4, 2019

ImportanceOfBeingErnest mentioned this pull request Aug 21, 2019

ValueError: cannot convert float NaN to integer #15093

Closed

efiring mentioned this pull request Sep 24, 2019

DivergingNorm is a misleading name #15336

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add DivergingNorm (again, again, again) #12419

Add DivergingNorm (again, again, again) #12419

jklymak commented Oct 6, 2018 •

edited

Loading

jklymak commented Oct 6, 2018

ImportanceOfBeingErnest commented Oct 6, 2018

jklymak commented Oct 6, 2018

ImportanceOfBeingErnest commented Oct 6, 2018

jklymak commented Oct 6, 2018

ImportanceOfBeingErnest commented Oct 6, 2018

QuLogic Oct 7, 2018

QuLogic Oct 7, 2018

phobson Oct 8, 2018

jklymak commented Oct 9, 2018

pharshalp commented Oct 9, 2018 •

edited

Loading

ImportanceOfBeingErnest commented Oct 9, 2018

jklymak commented Oct 9, 2018 •

edited

Loading

jklymak commented Mar 8, 2019 •

edited

Loading

anntzer Mar 13, 2019

jklymak Mar 13, 2019

timhoffm Mar 17, 2019

jklymak Mar 18, 2019

anntzer Mar 13, 2019

jklymak Mar 13, 2019

anntzer left a comment •

edited

Loading

jklymak commented Mar 13, 2019 •

edited

Loading

anntzer Mar 14, 2019

anntzer Mar 14, 2019

anntzer Mar 14, 2019

jklymak Mar 15, 2019

jklymak commented Mar 18, 2019

		assert norm.scaled() is True


		def test_DivergingNorm_scaleout_center():

		assert_array_equal(norm(vals), expected)


		def test_DivergingNorm_VminEqualsVcenter():

Add DivergingNorm (again, again, again) #12419

Add DivergingNorm (again, again, again) #12419

Conversation

jklymak commented Oct 6, 2018 • edited Loading

PR Summary

PR Checklist

jklymak commented Oct 6, 2018

ImportanceOfBeingErnest commented Oct 6, 2018

jklymak commented Oct 6, 2018

ImportanceOfBeingErnest commented Oct 6, 2018

jklymak commented Oct 6, 2018

ImportanceOfBeingErnest commented Oct 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jklymak commented Oct 9, 2018

pharshalp commented Oct 9, 2018 • edited Loading

ImportanceOfBeingErnest commented Oct 9, 2018

jklymak commented Oct 9, 2018 • edited Loading

jklymak commented Mar 8, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anntzer left a comment • edited Loading

Choose a reason for hiding this comment

jklymak commented Mar 13, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jklymak commented Mar 18, 2019

jklymak commented Oct 6, 2018 •

edited

Loading

pharshalp commented Oct 9, 2018 •

edited

Loading

jklymak commented Oct 9, 2018 •

edited

Loading

jklymak commented Mar 8, 2019 •

edited

Loading

anntzer left a comment •

edited

Loading

jklymak commented Mar 13, 2019 •

edited

Loading