Skip to content

Introduce new Tableau colors #12009

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

ImportanceOfBeingErnest
Copy link
Member

@ImportanceOfBeingErnest ImportanceOfBeingErnest commented Sep 3, 2018

PR Summary

As was mentionned by @josesho in #11927 (comment),
Tableau changed their default categorical colors in version 10 of their software.

image

Those colors are also available in Vega as "tableau10". Similarly a new set of 20 colors got introduced.

The new colors are less saturated and are hence pretty useful for colorzing larger areas in plots, e.g. for bars, pies etc. I would hence propose to add them to matplotlib as well.

There are two things to note here, for which I would also like to have feedback:

1. Naming

Since the default matplotlib (=old tableau) colors are already named tab:..., one cannot use this name. I am currently using tabx instead, due to the latin number X being 10 and the colors being introduced in version 10 of tableau.

  • tabx:colorname for the individual colors
  • The respective colormap is named tabx10 and tabx20

I find this not ideal, so I would ask for better ideas for naming.

2. Color order

The new colors' hues are in a different order than the old ones and hence the matplotlib default cycler. You see the original order in the left column in the picture below. In the right column I reordered them to match with the matplotlib defaults. Two options:

i. Keep the Tableau - order. Advantage: Consistency with "Tableau" and "Vega". (this is the current status of this PR.)
ii. Change to the matplotlib order. Advantage: Easy and consistent replacement for existing plots. Red is not next to orange (which looks strange with 3 color plots).

image

PR Checklist

  • Has Pytest style unit tests
  • Code is Flake 8 compliant
  • New features are documented, with examples if plot related
  • Documentation is sphinx and numpydoc compliant
  • Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
  • Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

Copy link
Member

@jklymak jklymak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine except for the name. When tableau 12 comes out will we call the palette tabxii? If we are to have all these palettes I wonder if we need to think how they are organized.

@@ -3,7 +3,7 @@
Color Demo
==========

Matplotlib gives you 8 ways to specify colors,
Matplotlib gives you 9 ways to specify colors,

1) an RGB or RGBA tuple of float values in ``[0, 1]`` (e.g. ``(0.1, 0.2, 0.5)``
or ``(0.1, 0.2, 0.5, 0.3)``). RGBA is short for Red, Green, Blue, Alpha;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’d say tab10:

Are these all really different “ways” to specify colours? If we are to have all these palettes I suggest specifying a palette is another way and then list all the palettes as a sub list.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and then list all the palettes as a sub list

Yes that is a very good idea.

@ImportanceOfBeingErnest
Copy link
Member Author

ImportanceOfBeingErnest commented Sep 3, 2018

I’d say tab10:

That is not really possible. The current default colors are named tab: ... and the corresponding colormap is "tab10". If we now name the new tableau colors tab10: ... that leads to confusion. Also I would really like to name the palette and colormap consistently, but that would lead to the new colormap "tab1010" and "tab1020". (urgh!)

@jklymak
Copy link
Member

jklymak commented Sep 3, 2018

Ok my knee jerk reaction is that this can get out of hand and we need some way to organize all these string-enabled colours since these aren’t just style files. Someone had solarized colours the other day and there are endless other lists of colours out there. Maybe this is a good chance to step back and think about how we manage these. Particularly as this one is so close to a previous one and has the same name

@ImportanceOfBeingErnest
Copy link
Member Author

Actually, for me personally it would suffice to create a respective colormap. I never used any of the named colors with a prefix before.

In view of #10840 and some of the suggestions therein I could well imaginge to have a single object Colormap which stores the colors and from which you may receive a cycler, as well as a palette of named colors.

@jklymak
Copy link
Member

jklymak commented Sep 3, 2018

You can get the RGB values of a listed colormap from cmp.values if that helps.

I added this to the weekly call agenda as a bit of discussion may help. Not sure the call will happen this week

@ImportanceOfBeingErnest
Copy link
Member Author

It's cmap.colors, but yes that's how I would get any color, or calling the colormap with integers. But that is not so obvious, so I can see how the palette:name notation is helpful for people.

The idea for organizing colors from above would be to have a single BunchOfColors object, which stores the colors. That could be a slightly enhanced Colormap with a .names attribute.

class Colormap:
    def __init__(self):
        name = ""
        names = None

    def get_named_colors():
         if self.names:
            names = [self.name+":"+n for n in self.names]
            return dict(zip(names , self.colors))

c = Colormap()
c.name = "tab"
c.names = ["blue", "orange", "green", ...]

One could read in the colors into names and then inside the colors.py fill the _colors_full_map with those key value pairs,

for cmap in allcmaps:
    if cmap.names:
        _colors_full_map.update(cmap.get_named_colors())

such that for any colormap with names in it there will be all colors in the palette notation available.

@timhoffm
Copy link
Member

timhoffm commented Sep 3, 2018

I don't have the time to get deeper involved here right now.

We have two only loosely related issues here:

  1. In what structure can we reasonably represent a sequence of discrete colors?
  2. How do we name these sequences and individual colors therein?

Issue 1 is certainly worth a thorough design discussion. Currently, we use a ListedColormap, which I don't think is a good representation (c.f. #10840). Maybe one has to have even a dedicated entity like Palette or ColorSequence, which is separate from cycler and Colormaps. Anyway, I don't want to go into this discussion now, and I don't think we have to for the present case.

Issue 2 is the naming of the palettes (or whatever we want to call it). Unfortunately, the naming is not quite consistent. We have:

  • "Pastel1" and "Pastel2", where the number is just an arbitrary discriminator
  • "Dark2", but not "Dark1"
  • tabNNN, which tries to add some semantics to the number, but even that's just half-hearted. The number gives the total number of entries in the palette. However, "tab20" has 10x2 colors, whereas "tab20b" and "tab20c" have 5x4 colors. That's a significant semantic difference.

Disregarding the existing names for a moment, I would say, that a naming pattern [name] and optionally [name]-[subtype] is reasonable. This could look like:

tableau, tableau-10x2, tableau-5x4, tableau-5x4-dark

The new colors would then be

tableau10, tableau10-10x2, tableau10-5x4, tableau10-5x4-dark

One option would be to just introduce the latter. Another option would be to deprecate the names tab, tab10, tab20 and replace them by the former.

Note: This change is independent of if we want to move on to something different than a ListedColormap. The ListedColormap will do for the moment. Also note that you can still consistently define color names like tableau10:blue or tableau10-5x4:orange2.

@phobson
Copy link
Member

phobson commented Sep 5, 2018

As much as I love the clarity of tableau10-5x4, I want to make sure that we don't overthink this.

image

I think a simple numbering scheme would work. For instance, Set1, Set2 have different numbers of colors and we don't call that out in there names. So I don't think that tableau20 and tableau20b need to be distinguished further -- though the missing "a" suffix on tableau20 does make my eye twitch a bit ;)

@ImportanceOfBeingErnest
Copy link
Member Author

@phobson Do you have a suggestion based on the above for the new tableau colormap/palette?

Also, what do people think about the second point - Color order?

@phobson
Copy link
Member

phobson commented Sep 6, 2018

@ImportanceOfBeingErnest I could go for "tab10b" or "tab10_v2" or "tab10_2018"

none of them are great, but at least they're not TableauColorFactoryQuadLevelFactoryBeans

@timhoffm
Copy link
Member

timhoffm commented Sep 6, 2018

A simple numbering scheme can indeed work. However,

  1. we already have a half-semantic one with tab10 and tab20. Actually, that naming was really confusing to me when I saw it the first time.
  2. we still need some name/subtype pattern. This may be [name][subtype], which would be compatible with the current naming. The only disadvantage is that there is no clear separation of the subtype, which is exactly the problem with tab10 being either subtype 10 of tab or tab10 without a subtype.

My goal is to have some sort of consistency in the naming pattern so that users don't stumble over it. I don't think this is overthinking. If the original names would just have been tab1, tab2, tab3, tab4. That would have been ok.

I strongly advise against either of "tab10b" or "tab10_v2" or "tab10_2018". We should at least keep the naming pattern consistent between the old and new tab variants. (Would we then have tab10b20b?).

I think adding a separation char between the name and subtype is still the best thing to do. If we don't do that, the original tabx variants are the second best option (and hoping that there will not be a tabular12 colormap in the future).

Concerning order, we should go for the original order of the colormap. The designers have made that change intentionally and we should stick with their decision for the Tableau10 colormap. A separate question is the default color cycle in matplotlib. Is it correct, that we currently use the tableau colors in the color cycle? I would not change these in a minor release, i.e. that's a discussion for matplotlib 4.0.

@ImportanceOfBeingErnest
Copy link
Member Author

The old tableau colors are indeed what is used in the matplotlib color cycle and they are the colors provided in tab10 colormap and as tab: ... named colors.

Those go under the name "category10" in vega. What matplotlib calls tab20, tab20b etc. goes under the name of category20, category20b etc.

All other categorical colormaps' names (Set1, accent, Paired...) are identical to vega. Vega names the new Tableau colors tableau10 and tableau20. Using those would of course be an option, although tab10 (old colors) and tableau10 (new colors) may sound more like the one is an abbreviation of the other?

Just to be clear here, I'm not proposing to change the default color cycle here, nor to rename any of the existing colormaps because that always causes some unnecessary friction.

@phobson
Copy link
Member

phobson commented Sep 6, 2018

Part of me thinks that the best course of action here is "do nothing" and defer people to https://jiffyclub.github.io/palettable/tableau/ and reach out to Matt to see if we can help keep palettable up to date.

@NelleV
Copy link
Member

NelleV commented Sep 7, 2018

We should also consider if we want that many variant of the same color palette. While I think having options is good, having too many options hampers usability.

@tacaswell
Copy link
Member

We should also get explicit approval from Tableau before we merge this (as we did for adding the current set).

We do not ship parula because Mathworks has asserted IP over it.

@tacaswell tacaswell added this to the v3.1 milestone Sep 7, 2018
Copy link
Member

@tacaswell tacaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval from Tableau to include these colors

Anyone can clear this once we have permission.

@ImportanceOfBeingErnest
Copy link
Member Author

Alright, so before asking for permission one should of course be clear on whether those colors should actually be added. There currently seem to be some objections and it would be quite embarrassing to first obtain permission and later decide against.

@jklymak
Copy link
Member

jklymak commented Sep 11, 2018

My vote is that these look nice, but don't bring enough new to the table to be added to the existing colors. In particular, I don't think they merit another bunch of named colors (i.e. that get string names).

Per the discussion on the call:

  • it'd be nice to have a way to register named colors the way we register colormaps. That way, if a library wanted to define a bunch of new named colors (i.e. 'tab2075:hot-pink') they could do so and still give users the ability to specify that color the way we specify 'xkcd:sky blue'. (I think thats just a public interface to _colors_full_map.update, but I'm not 100% sure).
  • the colormap part of this could be part of such a library or maybe submitted to palettable.

@tacaswell
Copy link
Member

it'd be nice to have a way to register named colors the way we register colormaps. That way, if a library wanted to define a bunch of new named colors (i.e. 'tab2075:hot-pink') they could do so and still give users the ability to specify that color the way we specify 'xkcd:sky blue'. (I think thats just a public interface to _colors_full_map.update, but I'm not 100% sure).

This was a beard-crumb we left during the 2.0 discussions that we did not follow up on yet. I think it should look something like def register_color_namespace(prefix: str, colors: Dict[str, Tuple[float, float, float, float]]). One concern would be how to handle the import issues if you want to use these colors in rcparams (and @anntzer will say just make them .py files ;) ).

@jklymak
Copy link
Member

jklymak commented Sep 11, 2018

... that looks great, though I'm not sure I like the idea of leaving a "beard-crumb"...

@ImportanceOfBeingErnest
Copy link
Member Author

For anyone looking to get the Tableau colors into matplotlib, I suppose the recommended way is

_tabx10colors = [
    "#4e79a7",  # blue
    "#f28e2b",  # orange
    "#e15759",  # red
    "#76b7b2",  # cyan
    "#59a14f",  # green
    "#edc948",  # yellow
    "#b07aa1",  # purple
    "#ff9da7",  # pink
    "#9c755f",  # brown
    "#bab0ac",  # grey
    ]

_tabx10names = ['tabx:blue', 'tabx:orange', 'tabx:red', 'tabx:cyan', 
                'tabx:green', 'tabx:yellow', 'tabx:purple', 'tabx:pink',
                'tabx:brown', 'tabx:grey']

import matplotlib.colors as mcolors
_tabxdict = dict(zip(_tabx10names, [mcolors.to_rgba(c) for c in _tabx10colors]))
mcolors.get_named_colors_mapping().update(_tabxdict)

(I don't see that much of a need for a registration API if all it's doing is adding some characters and a : in front of the name.)

@timhoffm
Copy link
Member

You can simply

_tab10colors = {
    'tabx:blue': "#4e79a7",
    'tabx:orange': "#f28e2b",
    'tabx:red': "#e15759",
    'tabx:cyan': "#76b7b2", 
    'tabx:green': "#59a14f",
    'tabx:yellow': "#edc948",
    'tabx:purple': "#b07aa1",
    'tabx:pink': "#ff9da7",
    'tabx:brown': "#9c755f",
    'tabx:grey': "#bab0ac",
}

import matplotlib.colors as mcolors
mcolors.get_named_colors_mapping().update(_tab10colors)

Though I'm not sure if that should be allowed. Why are you calling mcolors.to_rgba(c)?

And that's part of the problem I see. You shouldn't be able to mess with the color name table directly. There's no clear definition of the format. While that could be added, more importantly, there's no input validation. I can do

mcolors.get_named_colors_mapping()['blue'] = '#zzzzzz'
plt.plot([1, 3, 2], color='blue')

resulting in ValueError: Invalid RGBA argument: 'blue' with a lengthy call stack. The message is not exactly helpful, and in particular it should error out when I try to define a color, not when I use it.

Not having a registration API also means, that we get tied to the current implementation.

@jklymak jklymak added this to the v3.2.0 milestone Feb 26, 2019
@tacaswell tacaswell modified the milestones: v3.2.0, v3.3.0 Sep 4, 2019
@QuLogic QuLogic modified the milestones: v3.3.0, v3.4.0 May 2, 2020
@jklymak
Copy link
Member

jklymak commented Jul 16, 2020

So I'll close this. What happened with the stylesheet repository? @story645?

@jklymak jklymak closed this Jul 16, 2020
@story645
Copy link
Member

There's user contributed colormaps https://github.com/pyviz/contrib_colormaps

For stylesheets in particular @dhaitz has been building a tools for folks that I'd love to see integrated somehow:
https://matplotlib-style-voting.herokuapp.com/
https://matplotlib-style-configurator.herokuapp.com/

@jklymak
Copy link
Member

jklymak commented Jul 16, 2020

OK, maybe the style sheets are too matplotlib specific and we need to make our own repository? Not that the pyviz repository has gotten a lot of take-up yet.

@story645
Copy link
Member

Eh...someone would have to maintain and advertise it so that there would be uptake... and we should possibly update our docs w/ links to both repos. We can discuss on the call.

@QuLogic QuLogic removed this from the v3.4.0 milestone Mar 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants