-
-
Notifications
You must be signed in to change notification settings - Fork 7.8k
MultivarColormap and BivarColormap #28454
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MultivarColormap and BivarColormap #28454
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
haven't wrapped my head yet around what the call functions are doing
lib/matplotlib/colors.py
Outdated
"""Get the color for masked values.""" | ||
return np.array(self._rgba_bad) | ||
|
||
def set_bad(self, color='k', alpha=None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest to make the new colormaps immuatble. This should have been the case with current colormaps as well. Instead of modifying the existing colormap, return a new colormap with modified properties; c.f. Colormap.with_extremes
, #14645 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is reasonable suggestion, but there is a practical complication.
MultivarColormap
contains a list of Colormap
objects, which themselves are not immutable.
(I implemented MultivarColormap.__getitem__()
as a way to get the individual constituent colormaps, because I believe this is useful functionality.)
If we make MultivarColormap
immutable while the constituent colormaps are not I'm afraid we will run into some trouble in this regard.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, this is not optimal. But it's not as bad as one may think. Since colormaps are not immuatable colormaps[name]
or get_cmap(name)
return a new copy to prevent users from modifying the builtin colormaps. This also means, if you provide these colormaps as input to MultivarColormap
and don't keep references, you're safe. Additionally, I find it important from an education perspective to not give the users the tools to modify existing colormaps, even if we cannot ensure full immutability. And finally, we may be able to move towards immutable colormaps one day and thus should not put additional obstacles in our way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just so we are clear, this is the kind of use case I am envisioning:
cmaps = matplotlib.multivar_colormaps['2VarAddA']
norm_growth = matplotlib.colors.Normalize(vmin=2, vmax=10)
norm_capita = matplotlib.colors.Normalize(vmin=50, vmax=110)
p0 = matplotlib.collections.PatchCollection(patches, cmap=cmaps[0], norm=norm_capita)
p1 = matplotlib.collections.PatchCollection(patches, cmap=cmaps[1], norm=norm_growth)
p2 = matplotlib.collections.PatchCollection(patches, cmap=cmaps, norm=(norm_capita, norm_growth))
...
Where two datasets are shown independently, and also together, so that correlations can (ideally) be more easily identified. In order to use the same colormap both to visualize data independently, and together the constituent Colormap
objects of the MultivarColormap
object needs to accessed, i.e. via:
def __getitem__(self, item):
return self.colormaps[item]
Are you thinking we should change this to:
def __getitem__(self, item):
return self._colormaps[item].copy()
Where the member variable has been made private and we only return a copy.
With these changes we can then make the MultivarColormap
objects immutable.
(PS: I would like to use the above figure as part of an example in a later PR)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not exactly what I meant. I'm merely suggesting not to add any in-place modifying functions.
The copy comments were more to explain what happens with colormaps currently, and that you be don't have to be too concerned that somebody changes the input s you use under you. I haven't thought about extracting colormaps from the multivar colormaps. In particular here, I'd not make a copy because the colormaps are logically linked.
On a general note, I find this example difficult to follow; e.g. Washington is light grey. That this is a combination of saturated blue and saturated brown is hard to realize. This multivar approach has the fundamental problem that it can produce colors that are far from participating colormaps. The degree of the problem depends on the data and on the chosen colormaps. I feel for this visualization a bivar mapping like is more suited. Here one can easily see: black=both low, white=both high, orange/blue=one high and one low.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This discussion got a bit sidetracked.
Regarding the original question, I removed all set_
functions and introuduced with_extremes()
for both BivarColormap and MultivarColormap.
365df72
to
d11d47c
Compare
These look great! I'll take a closer look soon, but here's some info that I talked with folks at SciPy2024 about:
|
😄 I think we try to save the discussion on specific colormaps for a later PR, but the Millennium colormap sounds to me like the kind of thing to include. For now, there is |
This is a really nice PR. In addition to the above, I think the only thing that I would say should be super clarified down the line is exactly how two 'clashing' 1D colour maps can be smashed together to always produce in-range values. For instance A maps to [0, 0, 199] and B maps to [0, 0, 205] leading to [0, 0, 404] in 'Add' mode. I'm a little concerned that the existing implementation here can lead to low-contrast and low-brightness images (instead of using brightness-preserving blend modes). |
This is certainly an issue 😅, but it quickly becomes complicated. While it is perhaps not ideal to simply truncate the colors, it is not clear to me what the alternative is. That said, if the colormaps are designed with this in mind, it becomes less of an issue.
In perceptually linear colorspace (CAM02-LCD) the "rhombohedron" formed by this colormap looks like this: I find that if we want to have colormaps with greater saturation, we have to compromise on the points above and/or represent colors in a different colorspace. I wold really like to do the color mixing in CAM02-LCD directly, but the transformation Alternatively, if we are willing to compromise on point 4. (No combination leaves sRGB space) above, we can get higher saturation: If the data is sparse (i.e. each image is mostly black and the different images are bright in different areas), as is the case in the image above, this tradeoff might be worth it. Fluorescent microscopy (shown above) and element maps from X-ray fluorescence are the two main applications I am aware of where color mixing of 3+ channels is routine, and both of these techniques frequently produce sparse data. (Let me know if you know of another domain frequently uses 3+ color channels) That said, I think perhaps we should rename the names of combination modes to clarify for the user what is happening.
This also makes it easier to add additional ¹ I would like to add an tutorial for how users may subclass PS: let me know if there are any other way of color mixing that I have not thought of that you want me to try. |
Yes, this is really tricky! I agree with some very carefully chosen colour maps, you can get around this problem, and you've done a great job doing that. The specific blending mode that I would strongly suggest is 'lighten'. This compares n items, and keeps the colour that is the lightest. This is super useful in cases where you have many colour maps (here, if I remember correctly, I was using a perceptually uniform colour map segmented into 7 pieces, each which is assigned to a specific density value which scales with its brightness in a small range), and don't want to pay the 1/n brightness price as this is a sparse situation. Individual density maps look something like this: More information on this kind of visualisation is available in the yt docs (though I did not use yt for this...) https://yt-project.org/doc/visualizing/volume_rendering.html Now, the fundamental problem with this is that it is at some level lossy. You lose information about the low-brightness items if they are 'hidden' behind high-brightness items. But I think this is an acceptable (and in many cases beneficial) choice that should be available as a core feature, and dramatically opens up the visualization possibilities. |
Like this? We can easily add this feature, and I think we should. At that point we can discuss what the name should be¹, and if we need to ship the feature with additional colormaps. ¹I would like to have |
9fe5508
to
b59850d
Compare
Yes, that looks perfect! Thank you, definitely happy to defer this to a later PR. Just thought I would point this out as it's really a critical feature for sparse datasets. |
One thing that I will say is that there are already well-understood names for all of these modes; I would strongly suggest sticking with them. They are used in all design, drawing, and video editing programs and are well-understood. See e.g. https://www.clipstudio.net/how-to-draw/archives/154182 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
to me, this def looks good enough to get in so you can build on top of and may need a refactor later. Have some minor nits, only major question is all the special casing for like byte order and the like
self._clip((X0, X1)) | ||
|
||
# Native byteorder is faster. | ||
if not X0.dtype.isnative: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add the no-cover pragmas for now?
74b3fbf
to
44a3b00
Compare
sub_rgba, sub_mask_bad = c._get_rgba_and_mask(xx, bytes=False) | ||
sub_rgba = np.asarray(sub_rgba) | ||
rgba[..., :3] += sub_rgba[..., :3] # add colors | ||
rgba[..., 3] *= sub_rgba[..., 3] # multiply alpha |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reference / standard name for this method? It makes some sense, but is not normal compositing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This also has the property that if one colormap comes back with alpha=0
, then everything is 0, where I'm not sure makes sense. Should the alpha be pre-multiplied?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Truth be told, we have to treat alpha in some way, and in most use cases the implementation we choose does not matter. i.e. we can easily change this if you have an alternative expression you would like to use.
There was one use case I could think about: Two colorbars, one that is a color gradient, and another that is an alpha gradient. This implementation supports that use case, but I think it needs a bit more work to make it user-friendly.
This also has the property that if one colormap comes back with alpha=0, then everything is 0, where I'm not sure makes sense.
To me it makes sense that if alpha=0
on one colormap the result should have alpha=0
. The use case I am thinking of is a categorical colomap, and you want one of the categories to hide the element (i.e. missing data).
Should the alpha be pre-multiplied?
I'm not sure what this would imply, can you describe this with an equation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rgba[..., :3] += sub_rgba[..., :3] * sub_rgba[..., 3]
is what I meant by "pre-multiply".
I am not sure what the right expression is, but would like what ever it is to be well justified. Everything I found about blending was always 2 layers (not N) so I'm not sure if there an existing generalization available to us?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see any reason to do sub_rgba[..., :3] * sub_rgba[..., 3]
.
If we do this operation, then both [.5, .5, .5, 1]
and [1, 1, 1, .5]
produce the same color, and the former should always be preferred. I.e. this adds no new functionality to the multivariate colorbars.
To me, alpha has a unique use with regards to composition (i.e. layering different artists in a figure), and if we start mixing alpha and color, we interfere with this use of alpha.
The use case I have in mind is something like this:
import matplotlib.pyplot as plt
import numpy as np
import matplotlib
vals = np.zeros((256, 4))
vals[:, 3] = np.linspace(1,0, 256)
am = matplotlib.colors.ListedColormap(vals, 'alpha')
RB = plt.get_cmap('RdBu')
cmap = matplotlib.colors.MultivarColormap((RB, am), combination_mode='sRGB_add')
n = 100
x = np.linspace(-1,1, n)[np.newaxis, :]*np.ones((n,n))
y = np.linspace(-1,1, n)[:, np.newaxis]*np.ones((n,n))
r = np.sqrt(x**2+y**2)
a = np.random.random((n,n))*2-1
b = np.random.random((n,n))
fig, ax = plt.subplots()
ax.set_facecolor("k")
ax.contour(x, y, r, levels=10, colors='w', zorder=0)
cs = ax.pcolormesh(x, y, (a, b), cmap=cmap, zorder=2, vmin =(-1, 0), vmax = (1, 1))
fig.colorbar(cs.colorizer[0], ax=ax).set_label('RdBu')
fig.colorbar(cs.colorizer[1], ax=ax).set_label('Alpha')
Where different variables are shown using a diverging colorbar and lightness (alpha), and by using alpha, additional information can be shown behind the map (contour lines).
While this shows made-up data, I think you could make interesting maps with this. Consider for example a map of population density (alpha), median age (color) overlaid over a map of major highways.
|
||
cmaps = { | ||
name: LinearSegmentedColormap.from_list(name, data, _LUTSIZE) for name, data in [ | ||
('2VarAddA0', _2VarAddA0_data), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are pretty esoteric names; are they documented somewhere? Might even need a comment here as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@QuLogic thanks for all the comments
These are placeholders so that we have something to test with for this PR.
We should have a longer discussion about them in a later PR, which will also introduce more of them.
see this comment for a rough timeline #28428 (comment)
(I have been thinking we need sets from 2-8 colors)
At that point we should decide on a naming scheme.
I wrote a blogpost about their design here: https://trygvrad.github.io/multivariate-colormaps-for-n-dimensions/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So one of the reasons I suggested we maybe pull these in as dark mode cmaps #28454 (comment), and I'm okay w/ that happening in a later PR, is that we could then probably have more semantically meaningful names based on the individual maps.
0ae487f
to
a2fb40c
Compare
Also removed all set_ functions, and replaced them with with_extremes()
This renames the 'combination_mode' keywords from 'Add' and 'Sub' to 'sRGB_add' and 'sRGB_sub'. This change is intended to provide increased clarity if/when additional keywords are added.
Co-authored-by: Elliott Sales de Andrade <quantum.analyst@gmail.com>
f4361ed
to
a35bc29
Compare
@QuLogic I rebased this, do you want to take another pass and possibly approve this? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly typos and minor items, plus you should finish the _repr_png_
test noted above.
lib/matplotlib/colors.py
Outdated
patch : nparray of shape (k, k, 3) | ||
This patch gets supersamples to a lut of shape (N, M, 4) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See question above.
Co-authored-by: Elliott Sales de Andrade <quantum.analyst@gmail.com>
also allows SegmentedBivarColormap to take non-square patches. This was easier that testing that the input is always square.
b317208
to
dcf6f8a
Compare
@QuLogic thank you for looking at this again, I added a test for repr_png :) |
patch : np.array of shape (k, k, 3) | ||
This patch gets supersampled to a lut of shape (N, M, 4). | ||
patch : np.array | ||
Patch is required to have a shape (k, l, 3), and will get supersampled |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this supposed to be (k, k, 3)
not (k, l, 3)
? Sorry, still not sure what k
is; is it saying it must be square?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It used to be, but I updated the code so that it also works with non-square inputs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
only requesting changes b/c I don't know the status of cm.vectormappable and don't want this going in w/ docs referring to something non-existent.
based on code review by @story645
@story645 all your comments should be resolved now :) |
I can squash merge, but do you have a preference for the commit message. This is the default:
|
I was thinking simply
But I have no strong preferences, so do what is right for the project :) |
Creation of bivariate and multivariate colormap classes (colors.BivarColormap, colors.MultivarColormap)
PR summary
This PR has been in development during 2024 GSOC as a responese to #14168 Feature request: Bivariate colormapping.
This requires the introduction of multiple new classes, and changes to different parts of the codebase. It has therefore been suggested to introduce this functionality over multiple PRs, so that the changes are easier to review one of several PRs that will resolve #14168. See also PR #28428
The end goal of this is to allow for the following funcitonality:
The current class hierarchy can be visualized as follows:

This PR introduces classes for bivariate (2D) and multivariate (ND) colormaps, as shown in the figure below. It also includes tests for the new types of colormaps.
This PR does not contain the class cm.Vectormappable, nor does it contain the changes needed in the plotting functions, tests etc. These will follow in separate PRs, as outlined in the figure above. (a full implementation is available in the branch here)
PR checklist
¹This is the first of a series of PRs, and cannot by itself close #14168.