Skip to content

fire/ice blue/yellow bipolar colormap #6033

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
endolith opened this issue Feb 20, 2016 · 33 comments
Closed

fire/ice blue/yellow bipolar colormap #6033

endolith opened this issue Feb 20, 2016 · 33 comments

Comments

@endolith
Copy link
Contributor

There are several variations of this theme in existence:

I made one using bezier curves to reduce banding:

black middle

w 1

It does have a sharp angle at the midpoint causing a black band for 0, but that's intentional because it's for diverging data. (would be best with #1806) Here it is with a wavelet scalogram:

bipolar bezier

Like Ged Ridgway's, the midpoint brightness can be changed, with gray for surface plots, etc:

default

and color order flips if midpoint is brighter than gray:

n 1 bezier

Would this be good for inclusion, either as a (cleaned-up) function or as a single fixed map?

@Tillsten
Copy link
Contributor

The top-image is playing tricks with my eyes. I could really use it, i use 'bwr' all the time and wouldnt mind a little bit more color dynamic. (Time-resolved difference spectroscopy)

@tacaswell tacaswell added this to the 2.0 (style change major release) milestone Feb 20, 2016
@tacaswell
Copy link
Member

I am tentatively 👍 on including the black center and white center version over a single generation function.

The pros of just including the maps is that the cognitive load for users is low, but the con is low flexibilty.

The pro of adding the function is high flexibility, but also very high cognitive load / low discoverability. If you have a nice function to generate these maps, it might be better to integrate that into viscm (@njsmith @stefanv)

How do these colormap fare under the in luminosity linearity / color blind metrics?

For reference, these are the diverging color maps we currently ship:

image

@njsmith
Copy link

njsmith commented Feb 20, 2016

This is just designed by picking solve corners of the RGB cube and interpolating between them, which is not a design method that I'm particularly fond of -- but matplotlib has all kinds of more or less well designed colormaps in it so I'm not sure what the bar is. It would certainly make sense to at least stick it through viscm view.

@endolith
Copy link
Contributor Author

Well, if it were included as a function, it would be similar to the existing cubehelix function. Generates a fixed map out of the box, but can be accessed as a function if someone wants to generate a variation.

... Though now that I look at it, cubehelix isn't as easy to use as I expected. I thought you could do from matplotlib._cm import cubehelix and then cubehelix() would produce a LinearSegmentedColormap directly, but it only produces the RGB curves and has to be processed by _generate_cmap first. Maybe there should be a user-accessible matplotlib.cm.cubehelix_function() or something that calls _cm.cubehelix() and _generate_cmap() for you.

This is just designed by picking solve corners of the RGB cube and interpolating between them

Yep. But cyan and yellow are similar in perceptual lightness so it works pretty well. Here it is in Lab/Lch space:

bipolar lab

Here's what happens in RGB space when the midpoint is changed:

bezier rgb cube

Ideally matplotlib would have colorspace conversion abilities built-in and then we could do maps like this and color_scale as functions.

@endolith
Copy link
Contributor Author

endolith commented Feb 20, 2016

got viscm working:

viscm bipolar black center

@efiring efiring modified the milestones: 2.1 (next point release), 2.0 (style change major release) May 2, 2016
@efiring
Copy link
Member

efiring commented May 2, 2016

@endolith Maybe use viscm to flatten out the perceptual deltas? I really want to see a better set of colormaps like this made readily available for mpl--which might be by direct inclusion, or via an import-- but I don't want it to hold up 2.0, so I am moving the goalpost.

@tacaswell tacaswell modified the milestones: 2.1 (next point release), 2.2 (next next feature release) Oct 3, 2017
@endolith
Copy link
Contributor Author

FYI https://github.com/bokeh/colorcet provides 3 perceptually-smooth bipolar black-center maps now:

bky:

bky

bkr:

bkr

gkr:
gkr

vs my bezier bipolar:
bipolar

@choldgraf
Copy link
Contributor

Is anything in the license preventing us from bringing them into Matplotlib?

@endolith
Copy link
Contributor Author

License is CC-BY-4.0, and BSD does not require attribution? So I'm not sure. https://opensource.stackexchange.com/a/344/9996

@endolith
Copy link
Contributor Author

I got viscm working again

(for my own future reference:

pip install -U git+https://github.com/matplotlib/viscm

...

import viscm
viscm.gui.viscm(bipolar(neutral=0, lutsize=1024))

)

bky:
viscm bky

bkr:
viscm bkr

gkr (not for color-blind obviously):
viscm gkr

bipolar(neutral=0):
viscm bipolar

bipolar(neutral=1):
bipolar neutral 1

bipolar(neutral=0.3):
bipolar neutral 0 3

@choldgraf
Copy link
Contributor

Thanks for the viz! As always the plots I like the most are the ones least-perceptually-flat haha

re: Licenses, I think that we should be fine as long as we include that license here: https://github.com/matplotlib/matplotlib/tree/master/LICENSE does that sound correct @tacaswell ? Licenses are still mysterious to me...

@QuLogic
Copy link
Member

QuLogic commented Aug 11, 2018

BSD does not require attribution

That is not true; BSD requires reproducing both the copyright notice and the license.

@choldgraf
Copy link
Contributor

Ah so we need to include the "Anaconda" copyright language in matplotlib somewhere, as well as the license. That it?

@tacaswell
Copy link
Member

Yes, the licences files go in https://github.com/matplotlib/matplotlib/tree/master/LICENSE and the source needs to be noted is the whats_new, in any docstrings, and a comment where they are defined.

@tacaswell tacaswell modified the milestones: needs sorting, v3.1 Aug 11, 2018
@endolith
Copy link
Contributor Author

endolith commented Aug 12, 2018

Also found https://github.com/bastibe/twilight

twilight

and figured out how to use viscm edit, and tried to do the same thing to make a "more correct" version of bipolar, but I think it looks worse, with almost gray colors and visible halos/banding:

viscm edit version

bezier RGB bipolar:

bipolar

viscm bipolar:

hotcold first attempt

At high lightness I can get it to hug the chroma wall, since hue is changing:

2018-08-11 23_24_36-viscm editing _ new cm

but the way viscm's curve works, I can't get it to do the same at low lightness, since the hue is constant, so it descends into the low chroma gray muck in the middle:

2018-08-11 23_25_21-viscm editing _ new cm

In other words, there are multiple curves in 3D that have the same 2D top-down projection, but viscm only lets you adjust the 2D projection?

@njsmith
Copy link

njsmith commented Aug 12, 2018

In other words, there are multiple curves in 3D that have the same 2D top-down projection, but viscm only lets you adjust the 2D projection?

That's correct: given a 2D top-down projection there is only 1 possible 3D curve, if you add the constraint that you want the colormap to remain perceptually uniform when translated into greyscale, which viscm does. Of course you might say, well, who cares about printing in greyscale, here we're designing a diverging map with a V-shaped lightness curve, so anyone printing in greyscale is screwed anyway. And there's some truth to that. But you still should care about lightness uniformity at least a little, since it turns out that even when viewing images in color, people mostly attend to the lightness component, and ignore the hue/saturation – and crucially, this effect is not fully accounted for by the models we use to estimate perceptual uniformity. (Technically, IIRC, the issue is that the visual system is becomes relatively more sensitive to lightness and less sensitive to hue/saturation at higher spatial frequencies, and perceptual delta models like CAM02-UCS and CIELAB are all calibrated to match human performance on super-low spatial frequencies. And most data varies over space, that's why we need a colormap :-).) By making the colormap perceptually uniform in hue/saturation and perceptually uniform in lightness separately, we get a colormap that remains perceptually uniform at all spatial frequencies.

At least in theory. In practice, small deviations from perceptual uniformity are probably not that big a deal, and all these models are approximations anyway, so it's not clear how much we should really care about these details.

@endolith
Copy link
Contributor Author

endolith commented Aug 12, 2018

@njsmith

given a 2D top-down projection there is only 1 possible 3D curve, if you add the constraint that you want the colormap to remain perceptually uniform when translated into greyscale, which viscm does.

What I mean is that you can have multiple 3D curves that are identical when projected to 2D from above, and have the same lightness uniformity (the same linearly-spaced lightness along the black-to-white Z axis), but differ in how far out they go in the chroma radius. Instead of being at the yellow dot, I want it to be closer to the green dot (while maintaining the same lightness and hue):

2018-08-11 23_25_21-viscm editing _ new cm green dot

Obviously it's hard to describe 3D space curves in text, but you can see plots of color that hugs the RGB wall here, with two different philosophies: https://github.com/endolith/complex_colormap The low-lightness regions still have high chroma, they don't become gray.

The constant-chroma version follows this curve of chroma vs lightness: https://flic.kr/p/24QbbJG but around lightness of 30, the curve created by viscm is lower in chroma than it could be, because I can't describe the curve's shape in 3D, only 2D projection.

@njsmith
Copy link

njsmith commented Aug 12, 2018

What I mean is that you can have multiple 3D curves that are identical when projected to 2D from above, and have the same lightness uniformity (the same linearly-spaced lightness along the black-to-white Z axis), but differ in how far out they go in the chroma radius.

I don't understand what you mean. The chroma radius is the radius axis in the "2D from above" projection, so if they're identical when projected to 2D from above, then they're also identical in how far out they go in the chroma radius, by definition.

The "2D from above" project determines which (hue, chroma) values the curve passes through, but not when it does so – you can stretch or squish how it maps onto the [0, 1] data span. But if you uniform lightness spacing, and you want the final curve to be perceptually uniform in 3d space, then it implies that the points along the 2d curve also have to be uniformly spaced.

BTW, from complex_colormap's README:

Magnitude is mapped to lightness and phase angle is mapped to hue in a perceptually-uniform color space (previously LCh, now CIECAM02's JCh).

CIECAM02 JCh is not a perceptually uniform color space.

@endolith
Copy link
Contributor Author

The chroma radius is the radius axis in the "2D from above" projection, so if they're identical when projected to 2D from above, then they're also identical in how far out they go in the chroma radius, by definition.

I've been having a lot of trouble trying to generate the curves programmatically, so I'll draw a picture :)

Here is the view from above, in Jab/JCh space, with a path going from black (0,0,0) to blue (RGB 0,0,1 = JCh 21, 90, 258°), in a linear fashion, so that hue does not change, but lightness and chroma do change:

from above

The dotted line is the plane along which this next graph is a cross-section:

from side

Here you can see that there are actually 3 different curves, which all look the same from above.

They are all sampled linearly along the J axis (black dots), so when converted to grayscale, they are all identical.

However, they differ in the amount of chroma they have at each point in the curve.

  • The "g" plot mostly stays along the axis of rotation, so it's mostly shades of gray, before veering out to colorful blue at the last second.
  • The "L" curve is linearly-interpolated, and passes through a grayish dark blue region on its way to saturated light blue. I know viscm doesn't use linear interpolation, but it's following a similar path, passing through the grey unchromatic region more than it needs to.
  • The "m" curve tries to hug the wall of the RGB cube, increasing chroma as much as possible while still being smoothly-changing and convertible to RGB, so it's colorful even in the darkest region. This is the kind of curve I want for the "bipolar" colormap.

CIECAM02 JCh is not a perceptually uniform color space.

Oh! I must have misunderstood something. Which cylindrical coordinate colorspace is perceptually uniform? I had noticed that the lightness didn't seem linearly increasing, but I assumed it was from being displayed on an LCD vs the ideal illuminant scenario, etc.

@choldgraf
Copy link
Contributor

Hey all - this is a super interesting conversation, though can we get a quick check-in on the actionable points for this issue so it doesn't get lost? It sounds like somebody could:

  1. Copy over the colormap rules from Bokeh
  2. Maybe do this for the other repo linked in comments above?
  3. Copy the appropriate licenses and copyright language.

Does that sound right, or is there anything else? (maybe we could also paste a list such as this in the top-level comment so it's clear how we can close this issue if somebody wants to take it on :-) )

@endolith
Copy link
Contributor Author

endolith commented Aug 12, 2018

@choldgraf Well I'm in the process of trying to re-design a color map similar to my original comment, except starting in a perceptually uniform space, as per comments #6033 (comment) and #6033 (comment) (The grayscale map is not symmetrical, for instance.)

Whether to include colorcet/twilight/etc maps can be a separate Issue?

@njsmith

CIECAM02 JCh is not a perceptually uniform color space.

I see now that viscm uses J'/K as the vertical axis, not J, so my "linear on J" attempts are not linear in viscm. Is that what you mean?

2018-08-12 15_29_54-figure 2

@jklymak
Copy link
Member

jklymak commented Aug 12, 2018

Pretty sure twilight was merged already.

@anntzer
Copy link
Contributor

anntzer commented Aug 12, 2018

twilight was merged in #6254, and I think we should really think about #6254 (comment) before adding any new colormaps, regardless of their merits. Copied here (from @njsmith):

I think matplotlib probably needs to come up with some general guidelines for how they want to handle adding new colormaps -- in the limit you get in a situation where you have hundreds of random colormaps used by one person each, each with a more or less clever name.

@njsmith
Copy link

njsmith commented Aug 12, 2018

@endolith Oh, I see. A background assumption I was making was that you want a perceptually uniform colormap, which means that in your final 3d curve all the points are equally spaced. Your "g" and "m" plots don't have this property – the spaces between the dots are quite visibly different at different parts of the curve.

Again, you could potentially argue that you don't want to stick strictly to this constraint, but you should at least realize when you aren't :-)

J'/K

Yeah, the CIECAM02 equivalent to Lab is CAM02-UCS, whose coordinates are J'/K, a', b'. (The authors of that paper did not think these names through.) This is a Cartesian coordinate system. The equivalent of LCh is... "CAM02-UCS, converted to cylindrical coordinates". It doesn't have its own name, but it's just the grade school rectangular/polar conversion.

@endolith
Copy link
Contributor Author

which means that in your final 3d curve all the points are equally spaced.

Oh, so specifically, you mean the Euclidean distance (in CAM02-UCS space) between each sample is equal?

Yeah, the CIECAM02 equivalent to Lab is CAM02-UCS, whose coordinates are J'/K, a', b'.

Ok, that's helpful, I'll try to figure this out.

@njsmith
Copy link

njsmith commented Aug 12, 2018

so specifically, you mean the Euclidean distance (in CAM02-UCS space) between each sample is equal?

Exactly.

@anntzer
Copy link
Contributor

anntzer commented Nov 7, 2018

There was some discussion some time ago about making it easier for people to generate their own packages for distributing custom colormaps (this way not everything has to go into matplotlib directly); having just had to write one (for my own use) I put together https://github.com/anntzer/matplotlib-cmap-template.

(Yes, I know about cookiecutter and I actually quite like using that myself, but in this specific case I thought the template was just simple enough to not pick up a dependency on a templating system. If there's demand I can still make a cookiecutter for it though...)

@choldgraf
Copy link
Contributor

very cool @anntzer , thanks for sharing!

@timhoffm
Copy link
Member

timhoffm commented Nov 9, 2018

@anntzer Nice! Two suggestions:

  1. I would call it register_cmap(). That's more explicit and in a package in which the colormap is not the main topic, it's more clear what this does.

  2. Not quite sure if the __init__ example is good. I'd rather not have these in production code as this is cutter. Also, it's not an ideal example because it does not demonstrate how the to use the lib from your external code (as opposed to the exsample in the readme).

@anntzer
Copy link
Contributor

anntzer commented Nov 9, 2018

I would call it register_cmap().

Done.

Not quite sure if the __init__ example is good.

Uh? You mean __main__?
I don't see it as an example, but as some mildly useful functionality ("what was this colormap again?"); the docs should live (IMO) in the README or the docstring.

@timhoffm
Copy link
Member

timhoffm commented Nov 9, 2018

Uh? you mean __main__?

Yes, sorry (auto-type in my fingers...).

Ok, let's say +/-0 on __main__

@jklymak jklymak modified the milestones: v3.1.0, unassigned Feb 7, 2019
@anntzer
Copy link
Contributor

anntzer commented Feb 7, 2019

My proposition is to close this, encourage @endolith to publish the cmap as an independent PyPI package (for example (but not necessarily, of course) relying on the package template mentioned just above), and possibly link it back in the docs.

@jklymak
Copy link
Member

jklymak commented Feb 7, 2019

I agree w/ this.... plus the new colormap tutorial hopefully makes making custom colormaps easier... https://matplotlib.org/tutorials/colors/colormap-manipulation.html#sphx-glr-tutorials-colors-colormap-manipulation-py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

10 participants