Add power-law normalization #1204

bgamari · 2012-09-05T00:29:09Z

Here is a first cut at introducing power-law normalization (e.g. gamma correction) to matplotlib.colors. Do others feel this would be a useful addition to matplotlib?

I should note that it's unclear if/how data which is offset from zero should be handled which gives me pause.

travisbot · 2012-09-05T00:43:22Z

This pull request fails (merged aa31ca7d into 0f9f85f).

dmcdougall · 2012-09-05T07:33:52Z

lib/matplotlib/colors.py

+            return vmax**gamma * pow(val, -gamma)
+
+    def autoscale(self, A):
+        '''


This is not a docstring. Use """ instead.

Good to know. I actually stole this code outright from LogNorm. I'll put together a patch fixing this class as well.

bgamari · 2012-09-10T16:27:05Z

How does this look now? Comments?

WeatherGod · 2012-09-10T16:34:47Z

I doubt this will make v1.2.x as the feature freeze has already happened. However, it will still be interesting to get this in for the next release. The todo list is: unit tests and new entry to "what's new". Would also be nice to add some examples to the gallery.

I, too, would also like to make certain any sort of ambiguities regarding normalizations that are offset from zero.

bgamari · 2012-09-10T19:32:58Z

No worries regarding not making v1.2.

I'll be sure to handle the tasks you mention in the next spin of the patch. I'm not really sure what the best way to handle offset data is. I implemented what I believe is the obvious approach in 42a8b47. Here I simply remove the offset and adjust the normalization appropriately. This gives,

 >>> from matplotlib.colors import PowerNorm
 >>> n = PowerNorm(1.0, -100, 100)
 >>> n(-100)
 0.0
 >>> n(0)
 0.5
 >>> n(100)
 1.0

Does this seem reasonable?

bgamari · 2012-09-11T18:24:28Z

Seeing as there were no objections to my handling of offset data, I squashed this commit as well as another fix into b277f87.

dmcdougall · 2013-01-18T02:35:28Z

lib/matplotlib/colors.py

+        vmin, vmax = self.vmin, self.vmax
+        if vmin > vmax:
+            raise ValueError("minvalue must be less than or equal to maxvalue")
+        elif vmin==vmax:


PEP8: Spaces needed around the ==.

dmcdougall · 2013-01-18T02:39:02Z

Let's try and get this into v1.3.

@WeatherGod Are you happy with the handling of offsetting?

dmcdougall · 2013-01-18T02:40:20Z

This needs a test, an entry in CHANGELOG and whats_new, an example, and the issues outlined above need to be addressed as well.

bgamari · 2013-01-18T22:03:56Z

Thanks for the comments! I'll try to incorporate them this weekend.

dmcdougall · 2013-01-22T02:11:37Z

lib/matplotlib/tests/test_colors.py

@@ -4,7 +4,7 @@

 from __future__ import print_function
 import numpy as np
-from numpy.testing.utils import assert_array_equal, assert_array_almost_equal


The SymLogNorm test uses the assert_array_almost_equal function. Removing this made the test fail because it couldn't find it.

bgamari · 2013-01-22T15:32:11Z

I apologize for the sad state of the previous branch. It was admittedly incomplete work and I hadn't managed to get the testsuite to run. Thanks for the suggestions.

dmcdougall · 2013-01-22T16:17:31Z

@bgamari No problem! That's why I'm here :)

bgamari · 2013-01-22T16:43:34Z

@dmcdougall, how do things look now? I think this should be in good shape now although if you have any suggestions for further additions to the whats_new contribution, I'd be happy to hear.

dmcdougall · 2013-01-23T04:33:59Z

examples/api/power_norm_demo.py

@@ -0,0 +1,27 @@
+#!/usr/bin/python
+
+from scipy.stats import norm


Matplotlib doesn't have scipy as a dependency, so we can't have this line in the example. On the plus side, it looks like this function isn't even used in the example.

Yes, this snuck in. Killed now

bgamari · 2013-11-27T15:15:10Z

@mdboom, indeed it does seem that the PowerNorm test fails on Travis. I'm really not sure why, though. The test succeeds for me and Travis doesn't seem to provide any additional diagnostics

mdboom · 2013-11-27T16:07:43Z

lib/matplotlib/colors.py

+            self.vmin = ma.min(A)
+            if self.vmin < 0:
+                self.vmin = 0
+                warn("Power-law scaling on negative values is ill-defined, clamping to 0.")


I think this needs to be warnings.warn, and is the cause of the Travis failure.

bgamari · 2013-11-27T16:51:09Z

@mdboom, good catch! We'll see how this branch fares.

tacaswell · 2013-11-27T18:03:03Z

lib/matplotlib/colors.py

+        self.vmin = ma.min(A)
+        if self.vmin < 0:
+            self.vmin = 0
+            warnings.warn("Power-law scaling on negative values is ill-defined, clamping to 0.")


bgamari · 2013-11-28T15:19:30Z

I've fixed the remaining PEP8 issues as well as the testcase. It seems that Travis approves.

pelson · 2013-12-05T10:24:47Z

lib/matplotlib/tests/test_colors.py

+
+    a = [-0.5, 0, 0.5, 1, 1.5]
+    pnorm = mcolors.PowerNorm(1.5)
+    assert_array_almost_equal(a, pnorm.inverse(pnorm(a)))


This is testing that the inverse method is working, but there is no test to check that power norm is actually doing the right thing. For example, I believe I could change line 65 from mcolors.PowerNorm(1.5) to mcolors.Normalize() and the test will still pass.

@bgamari Did this get addressed?

No, not yet.

pelson · 2014-01-09T15:05:50Z

Looks like a small test extension and a rebase would see this into v1.4.0. Is there any chance you could take a look @bgamari?

bgamari · 2014-01-22T02:57:36Z

How does this look?

tacaswell · 2014-01-22T03:26:28Z

Travis produces a divide by 0 error in this test.

tacaswell · 2014-01-22T03:32:02Z

lib/matplotlib/tests/test_colors.py

+    assert_array_almost_equal(norm(a), pnorm(a))
+
+    a = [0.5, 0, 2, 4, 8]
+    expected = [1/0, 0, 1./16, 1./4, 1]


The divide by zero error is explicit in the test. That does not seem right.....

Yep, sorry about that; I was still waiting for my tests to finish when I pushed this branch. The 1/0 should have been np.nan.

mdboom · 2014-01-27T18:33:32Z

@bagmari: It looks like this no longer merges cleanly. Would you mind rebasing against master?

bgamari · 2014-01-27T19:11:52Z

I believe I (finally) have the test issues sorted out and the branch rebased.

tacaswell · 2014-02-26T17:51:58Z

There is still a py3k issue:

ERROR: matplotlib.tests.test_colors.test_PowerNorm

----------------------------------------------------------------------

Traceback (most recent call last):

File "/home/travis/virtualenv/python3.3/lib/python3.3/site-packages/nose/case.py", line 198, in runTest

self.test(*self.arg)

File "/home/travis/virtualenv/python3.3/lib/python3.3/site-packages/matplotlib-1.4.x-py3.3-linux-x86_64.egg/matplotlib/tests/test_colors.py", line 62, in test_PowerNorm

assert_array_almost_equal(norm(a), pnorm(a))

File "/home/travis/virtualenv/python3.3/lib/python3.3/site-packages/matplotlib-1.4.x-py3.3-linux-x86_64.egg/matplotlib/colors.py", line 1184, in __call__

result[value < 0] = 0

TypeError: unorderable types: list() < int()

It looks like something needs to be explicitly made into an numpy array. I am very confused why this also does not happen on 2....

bgamari · 2014-03-10T15:38:30Z

Looks like we've finally converged.

tacaswell · 2014-03-10T16:03:00Z

doc/users/whats_new.rst

@@ -181,19 +189,6 @@ added. Furthermore, the the subplottool is now implemented as a modal
 dialog. It was previously a QMainWindow, leaving the SPT open if one closed the
 plotwindow.

-Cairo backends


This probably should not be removed.

damn you git rerere

bgamari · 2014-03-10T16:28:29Z

@tacaswell, thanks again for catching all of these mistakes. I think I probably owe you a case of beer or two by now.

Add power-law normalization

tacaswell · 2014-04-04T02:28:22Z

This should get a medal as an exceptionally long running but finally merged PR.

dmcdougall reviewed Sep 5, 2012
View reviewed changes

ghost assigned dmcdougall Jan 18, 2013

dmcdougall reviewed Jan 18, 2013
View reviewed changes

dmcdougall reviewed Jan 22, 2013
View reviewed changes

dmcdougall reviewed Jan 23, 2013
View reviewed changes

mdboom reviewed Nov 27, 2013
View reviewed changes

tacaswell reviewed Nov 27, 2013
View reviewed changes

pelson reviewed Dec 5, 2013
View reviewed changes

tacaswell reviewed Jan 22, 2014
View reviewed changes

tacaswell added the needs_revision label Mar 8, 2014

bgamari added 2 commits March 10, 2014 11:17

colors: Replace docstring ''' with """

9b29988

test_colors: Fix typo

ddd5899

tacaswell reviewed Mar 10, 2014
View reviewed changes

bgamari added 3 commits March 10, 2014 12:26

colors: Add power-law normalization

1b9fc6c

power_norm_demo: Initial commit

48998c1

hist2d: Add a note mentioning PowerNorm

d5343c0

tacaswell added a commit that referenced this pull request Apr 4, 2014

Merge pull request #1204 from bgamari/power-law-norm

d0250b4

Add power-law normalization

tacaswell merged commit d0250b4 into matplotlib:master Apr 4, 2014

QuLogic removed the status: needs revision label Nov 24, 2016

efiring mentioned this pull request Jan 11, 2018

PowerNorm: do not clip negative values #10234

Merged

		@@ -0,0 +1,27 @@
		#!/usr/bin/python

		from scipy.stats import norm

Add power-law normalization #1204

Add power-law normalization #1204

Conversation

bgamari commented Sep 5, 2012

travisbot commented Sep 5, 2012

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bgamari commented Sep 10, 2012

WeatherGod commented Sep 10, 2012

bgamari commented Sep 10, 2012

bgamari commented Sep 11, 2012

Choose a reason for hiding this comment

dmcdougall commented Jan 18, 2013

dmcdougall commented Jan 18, 2013

bgamari commented Jan 18, 2013

Choose a reason for hiding this comment

bgamari commented Jan 22, 2013

dmcdougall commented Jan 22, 2013

bgamari commented Jan 22, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bgamari commented Nov 27, 2013

Choose a reason for hiding this comment

bgamari commented Nov 27, 2013

Choose a reason for hiding this comment

bgamari commented Nov 28, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pelson commented Jan 9, 2014

bgamari commented Jan 22, 2014

tacaswell commented Jan 22, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdboom commented Jan 27, 2014

bgamari commented Jan 27, 2014

tacaswell commented Feb 26, 2014

bgamari commented Mar 10, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bgamari commented Mar 10, 2014

tacaswell commented Apr 4, 2014