This will make it easier to create custom backends, by simply installing the right file in the right place for Matplotlib to discover.

Can you clarify this statement (i.e. the goal of this PR)? Currently you can already make any module a custom backend (which is then selectable using the module://importable.backend.name syntax).

@anntzer sorry, I realize my PR comment is indeed missing some context.

This is hopefully the first PR of a bunch of PRs that would clean up the IPython-Matplotlib interface.

Currently, IPython provides a %matplotlib magic that allows using Matplotlib inside of Jupyter or from the IPython shell. This magic takes a backend name as argument and maps this name to the right Python module containing the backend (see the code). This worked well for several years, but it is completely inflexible. For example, we cannot rename the file containing the backend in ipympl (currently named nbagg because it started as a copy of nbagg), because it's hardcoded in IPython that it has to be named this way. It is also impossible for developers to come up with their own IPython-compatible backend, without changing IPython's code.

This PR tries to make this more modular, by enforcing the backends to provide a manifest that advertises the backend name, module, and if it's compatible with IPython (interactive). This way IPython and Matplotlib can dynamically discover available backends without importing them.

Ideally, this %matplotlib magic would be provided by Matplotlib. Because, as of right now, there is a big special case for Matplotlib in IPython. But I guess this is another discussion.

This comes after a discussion that happened with @tacaswell and @SylvainCorlay something like a year ago, where we discussed improving this logic.

Maybe @Carreau would have some ideas on this as well.

sorry, I realize my PR comment is indeed missing some context.

No worries :)

This is hopefully the first PR of a bunch of PRs that would clean up the IPython-Matplotlib interface.

That's a worthwhile goal too :)

Currently, IPython provides a %matplotlib magic that allows using Matplotlib inside of Jupyter or from the IPython shell. This magic takes a backend name as argument and maps this name to the right Python module containing the backend (see the code). This worked well for several years, but it is completely inflexible.

For example, we cannot rename the file containing the backend in ipympl (currently named nbagg because it started as a copy of nbagg), because it's hardcoded in IPython that it has to be named this way.

But that could fully be solved on IPython's side, no? IPython could just support %matplotlib module://ipympl.backend_nbagg (and if that's too long to type, we could e.g. reasonably just say that any dotted backend name gets interpreted as a fully qualified module name, and then it's just %matplotlib ipympl.backend or whatever ipympl wants to call it). (*)

It is also impossible for developers to come up with their own IPython-compatible backend, without changing IPython's code.

Actually this is possible, see https://github.com/matplotlib/mplcairo/blob/master/lib/mplcairo/_util.py#L15. (I'm not saying patching backend2gui is particularly nice, but it's possible, and a nicer but backcompatible API could be developed around that.)

This PR tries to make this more modular, by enforcing the backends to provide a manifest that advertises the backend name, module, and if it's compatible with IPython (interactive).

Per (*) I believe reusing module names as backend names is simple and sufficient enough. For example, you don't have to worry about two different modules claiming to provide the same backend name.

A priori the metadata for interactivity is the FigureCanvas.required_interactive_framework attribute.

This way IPython and Matplotlib can dynamically discover available backends without importing them.

The importance of avoiding import is unclear to me (but I may be missing something).

Ideally, this %matplotlib magic would be provided by Matplotlib. Because, as of right now, there is a big special case for Matplotlib in IPython. But I guess this is another discussion.

Agreed.

Thank your for your detailed answer :)

But that could fully be solved on IPython's side, no? IPython could just support %matplotlib module://ipympl.backend_nbagg (and if that's too long to type, we could e.g. reasonably just say that any dotted backend name gets interpreted as a fully qualified module name, and then it's just %matplotlib ipympl.backend or whatever ipympl wants to call it).

But my point is that it should be up to the backend to provide its name and the path to the Python module containing its implementation. It should not be IPython's job. It's a separation of concerns issue.

It seems to me that it should be up to Matplotlib to have a way to provide a list of available backends. And it should not be hardcoded anywhere, not in IPython, not in Matplotlib.

Actually this is possible, see https://github.com/matplotlib/mplcairo/blob/master/lib/mplcairo/_util.py#L15. (I'm not saying patching backend2gui is particularly nice, but it's possible, and a nicer but backcompatible API could be developed around that.)

This works indeed. But it's only a short-term solution. Also, it doesn't seem to me that backend2gui should be considered public API, so this feels like a very weak solution.

A priori the metadata for interactivity is the FigureCanvas.required_interactive_framework attribute.

Thanks! I missed that :)

The importance of avoiding import is unclear to me (but I may be missing something).

This is just a good thing to have IMO, to avoid importing unused libraries. If we can find a way to avoid importing all available backends it's better.

But that could fully be solved on IPython's side, no? IPython could just support %matplotlib module://ipympl.backend_nbagg (and if that's too long to type, we could e.g. reasonably just say that any dotted backend name gets interpreted as a fully qualified module name, and then it's just %matplotlib ipympl.backend or whatever ipympl wants to call it).

But my point is that it should be up to the backend to provide its name and the path to the Python module containing its implementation. It should not be IPython's job. It's a separation of concerns issue.

My point is that the backend name should exactly be the module name (with the exception of the builtin backends (and possibly the ipympl backends), which are grandfathered). See cbook._backend_module_name. Currently the backend name is the module name prefixed by module://, but we can reasonably change that (and support matplotlib.use("importable.name") instead of matplotlib.use("module://importable.name")) with no backcompat breakage whatsoever.

It seems to me that it should be up to Matplotlib to have a way to provide a list of available backends. And it should not be hardcoded anywhere, not in IPython, not in Matplotlib.

Why do you need to have this list available?
In particular, I would like to keep the possibility to dynamically generate backends (by creating namespace objects that just expose a FigureCanvas attribute, and inserting them in sys.modules). This may be useful e.g. if we get better separation of the GUI integration and rendering parts, so that in the future I could get e.g. a qt+cairo backend that dynamically uses qt for GUI integration and cairo for rendering (so we only need to write M GUI integrations + N renderers), rather than having to explicitly write all GUI+renderers (M*N) combinations.

Actually this is possible, see https://github.com/matplotlib/mplcairo/blob/master/lib/mplcairo/_util.py#L15. (I'm not saying patching backend2gui is particularly nice, but it's possible, and a nicer but backcompatible API could be developed around that.)

This works indeed. But it's only a short-term solution. Also, it doesn't seem to me that backend2gui should be considered public API, so this feels like a very weak solution.

OK, but AFAICT IPython only needs this dict because it doesn't know to read the required_interactive_framework attribute. If it learnt to do that, it could just ask matplotlib "give me the backend module" and itself read mod.FigureCanvas.required_interactive_framework.

The importance of avoiding import is unclear to me (but I may be missing something).

This is just a good thing to have IMO, to avoid importing unused libraries. If we can find a way to avoid importing all available backends it's better.

But why do you need to list them at all? The user says %matplotlib foo, requesting backend foo, IPython runs matplotlib.use("foo") (we could even make matplotlib.use learn about the grandfathered ipympl name), and gets a backend module from matplotlib (or an ImportError if no such backend exists), and can then get mod.FigureCanvas.r_i_f. Only the actually used backend is imported.

My point is that the backend name should exactly be the module name (with the exception of the builtin backends (and possibly the ipympl backends), which are grandfathered). See cbook._backend_module_name. Currently the backend name is the module name prefixed by module://, but we can reasonably change that (and support matplotlib.use("importable.name") instead of matplotlib.use("module://importable.name")) with no backcompat breakage whatsoever.

I understand. I honestly don't mind if they are not named. Doesn't it seem more user-friendly that backend can provide a name? Just for convenience.

Why do you think the backend name should be the module name?

In particular, I would like to keep the possibility to dynamically generate backends (by creating namespace objects that just expose a FigureCanvas attribute, and inserting them in sys.modules). This may be useful e.g. if we get better separation of the GUI integration and rendering parts, so that in the future I could get e.g. a qt+cairo backend that dynamically uses qt for GUI integration and cairo for rendering (so we only need to write M GUI integrations + N renderers), rather than having to explicitly write all GUI+renderers (M*N) combinations.

I see. But don't you think inserting to sys.module is a bit unclean? I see that those manifests would be annoying in this case. I wonder if we could find a more proper way for all this.

But why do you need to list them at all? The user says %matplotlib foo, requesting backend foo, IPython runs matplotlib.use("foo") (we could even make matplotlib.use learn about the grandfathered ipympl name), and gets a backend module from matplotlib (or an ImportError if no such backend exists), and can then get mod.FigureCanvas.r_i_f. Only the actually used backend is imported.

Fair point 👍🏼

Doesn't it seem more user-friendly that backend can provide a name? Just for convenience.
Why do you think the backend name should be the module name?

I maintain the mplcairo backends (https://github.com/matplotlib/mplcairo) and use("module://mplcairo.qt") (.base, .gtk, etc.) seems perfectly reasonable to me (perhaps modulo stripping out the module:// prefix, as suggested above)? What else would I want to name them?

I see. But don't you think inserting to sys.module is a bit unclean? I see that those manifests would be annoying in this case. I wonder if we could find a more proper way for all this.

It doesn't really have to be a module, we could just support use(<backend-like-object>) without going through sys.modules.

My take on this is that we cannot change the fact that there are hundreds of thousands of notebooks out there that make use of the matplotlib magic

%matplotlib inline  # qt, nbagg, widget

and should we make any change to how the matplotlib magics maps to backends, we have to make sure that we don't break these notebooks. In my mind, %matplotlib foobar, foobar is merely an abreviation for a module name. It should also work with the actual module name in the end...

What I think this PR brings is for the backends to provide their own abbreviation if any, so that the mapping from abbreviation to module name is maintained in the same place as the module name.

for the backends to provide their own abbreviation

Again, I believe that the already existing list of abbreviations should be grandfathered for backcompat purposes (if you don't want to hardcode the mapping in multiple places, we can reasonably add it to matplotlib as an (immutable) mapping), but that we should not support adding more abbreviations, and instead make people use module names instead. We don't have a metadata/plugin system to allow people to write import mpl or import skl instead of import matplotlib or import sklearn; I don't see why backend names should be different.

To be more precise, if people want to be able to write import mpl instead of import matplotlib, they can create a mpl.py in their sys.path with from matplotlib import * (actually that won't work because of subpackages, but let's ignore that for now); likewise if people want to write %matplotlib foo instead of %matplotlib some.long.module.name they can create a foo.py in their sys.path with from some.long.module.name import * -- once we change the %matplotlib magic to support module names, which is again something I would support.

We talked about this on the mpl dev call and came to the consensus that:

• under no condition should we change something that breaks existing %matplotlrib XYZ behavior because there are too many notebooks and too much muscle memory for that to be plausible
• the hard coded list of abbreviations -> modules should move from IPython to Matplotlib but the magic should still probably live in ipykernal. IPython can then either conditionally import the mapping back, or if mpl now owns it, we can handle the re-mapping in switch_backend.
• we should adopt @anntzer 's suggestion that %matplotlib unknown.module gets up-converted to module://unknown.module and processed as such (we can do that on the mpl side so that mean). I think this means that IPython will be able to blindly pass the backend through to pyplot.switch_backend and let mpl handle all of the normalization / aliasing / exception and I think gui fallback.

We already have a mechanism for the backends to report what GUI framework they need and if we discover that IPython is already imported reach back and set up the event loop integration so I think backend2gui can be fully removed!

I think the above is non-controversial, can be done quickly, and is a big step forward.

Below is my thoughts, not consensus from the call.

I can see two main upsides of auto-discovery: being able to list available backends and backends being able to provide an alias/metadata. %matplotlib --list currently exists, but we could move that to the current static list and add "If you have any third-party backends installed initialize %matplotlib module.name" or similar. I agree with @anntzer that adding aliases here is a bit superfluous if we stop requiring module:// .

That said, I could be talked into either or both of those being valuable enough to be worth implementing auto-discovery, but I would prefer if we did it through entrypoints and I think can be handled separately from the points above.

I think the %matplotlib magic has to continue to live in IPython as it needs to be available to users without any imports and our import time is bad enough you really do not want a unconditional import of even our top-level to make make it available.

Would that be a problem? People can also mistype regular imports. If you have nbag.py on your system it should be safe to import. We'd just raise afterwards because it does not provide a backend. If you have files that are dangerous to import, you likely have bigger issues. There are various tools that imports lots of stuff (AFAIR e.g. Sphinx, pytest, ...)

I agree with @timhoffm that if you have nbag.py in your import path that reformats your hard drive (or whatnot) on import, then you have much bigger issues than possibly accidentally importing it from matplotlib, so this is not a realistic threat model.

On another point, assuming that the end goal is to have the implementation of %matplotlib move to matplotlib itself (which I guess means it would map to calling some matplotlib.ipython_integration.matplotlib_magic() function), I think we could even just forget about the second argument? My understanding(?) is that matplotlib_magic() basically needs to call some GUI-framework-specific code to enable integration with IPython, but instead it could just register a flag and have matplotlib.use() both set up the backend (as it did before), and, if the flag is present, perform the integration? (IOW, we don't need two sources of truth about what backend is in use -- matplotlib.use() should be the sole way to do that.)

(Of course we can't break backcompat so still need to support a backend being passed to %matplotlib, but we can just map that to setting the flag and then calling use().)

the hard coded list of abbreviations -> modules should move from IPython to Matplotlib

The main argument I see against keeping an hard coded list somewhere is the ipympl example: we are currently stuck by IPython on the naming of the ipympl backend module (which was named backend_nbagg because it started as a copy-paste of the nbagg backend...).

Moving this hard coded list from IPython to Matplotlib would, unfortunately, not make this kind of issues easier. I would find it nicer/more sensible if it's the backend itself that advertises its alias, as mentioned by @tacaswell.

but I would prefer if we did it through entrypoints

I totally agree on this. I can try to come up with a PR adding such logic of discovering the entrypoints if you want.

The main argument I see against keeping an hard coded list somewhere is the ipympl example: we are currently stuck by IPython on the naming of the ipympl backend module (which was named backend_nbagg because it started as a copy-paste of the nbagg backend...).

I don't really understand what (if anything) changed here, compared to the previous discussion: numpy, for example, is stuck forever on having numpy/__init__.py (or at least a shim numpy.py); they cannot decide that they want to have an official alias as np (so that you could import np) without breaking backcompat.
In fact the situation is less bad for the backend alias name because there are few enough widely used backends that we could add manually add entries to the table if really needed (but better would be to just rename the module and keep the old one as a shim). (Again, I would not want to introduce new aliases, but we need to keep the old ones forever anyways.)

numpy, for example, is stuck forever on having numpy/init.py (or at least a shim numpy.py); they cannot decide that they want to have an official alias as np (so that you could import np) without breaking backcompat

NumPy could definitely decide to deprecate some modules and show a deprecation warning saying that those modules would be removed in a coming major release.
It is totally fine for users who directly depend on NumPy, they could have a NumPy version check prior to import or do a try/catch ImportError.

The problem here with the hard-coding of the aliases in IPython is that if ipympl wants to rename its module (which it should), then IPython will need to figure out the ipympl version that is installed in order to know which module the alias points to. This seems like an unreasonable situation. In my opinion, IPython should not even know what ipympl is, it does not depend on it in any way, note that it doesn't even depend on Matplotlib.

Having entry points would fix this, because ipympl would advertise to others which submodule contains the backend. ipympl would be the source of truth.

I would not want to introduce new aliases

Why not?

Being able to introduce new aliases without having to change IPython's code would actually be nice :) Being able to do %matplotlib foobar for the lesser known foobar backend instead of %matplotlib module://foobar.path.to.backend seems reasonable.

NumPy could definitely decide to deprecate some modules and show a deprecation warning saying that those modules would be removed in a coming major release.
It is totally fine for users who directly depend on NumPy, they could have a NumPy version check prior to import or do a try/catch ImportError.

But then that's not different from doing a deprecation dance on a backend name to reintroduce it under another backend name, and have users to try... catch around that (well, I don't know if you can wrap %matplotlib foo in a try catch but 1) perhaps you should and 2) perhaps that can be spelled something like get_ipython().enable_matplotlib("foo") or whatnot).

The problem here with the hard-coding of the aliases in IPython is that if ipympl wants to rename its module (which it should), then IPython will need to figure out the ipympl version that is installed in order to know which module the alias points to. This seems like an unreasonable situation. In my opinion, IPython should not even know what ipympl is, it does not depend on it in any way, note that it doesn't even depend on Matplotlib.

As stated previously, I am totally OK with moving the current alias list from ipython to matplotlib, so ipython would not need to know about ipympl.

Having entry points would fix this, because ipympl would advertise to others which submodule contains the backend. ipympl would be the source of truth.

Why not?

Because this breaks the simple modulename<->backendname equivalence, and I remain unconvinced that the benefits justify adding that machinery and the code that goes with it.

Being able to introduce new aliases without having to change IPython's code would actually be nice :) Being able to do %matplotlib foobar for the lesser known foobar backend instead of %matplotlib module://foobar.path.to.backend seems reasonable.

As stated previously, I (and @tacaswell too) am OK with supporting the spelling `%matplotlib foobar.path.to.backend, and then it's just up to you how deep you want to nest your submodule...

The hardcoded list "grandfathered" into Matplotlib suggested by @anntzer is not a practical solution because several of the backends listed in there are not core to Matplotlib (the inline and widget backends). A hardcoded list would lead to the same "locking" problem as we have now with respect to changing the package name in e.g. ipympl.

The approach proposed here by @martinRenou is the most practical in my opinion.

ok, I think a summary of the points here are:

• there has been some on-going work to tighten up the definition of what a "backend" is. For historical reasons a backend has been a module, but there is a push to make a backend a class (possible making it so FigureCanvas is the rosetta stone, see Standardize creation of FigureManager from a given FigureCanvas class. #18854)
• %matplotlib foo with at least the current aliases needs to work as-is for backends owned by Matplotlib
• any aliases -> backend mapping needs to live on the Matplotlib side (so %matplotlib can fully defer to plt.switch_backend)
• matplotlib.use(obj) and matplotlib.use('module://XYZ') both should work (only the later currently works, but we have a proof-of-concept for using backend objects directly in https://github.com/matplotlib/mpl-gui/blob/5be9b47a6afb018e79c502e5c8c9211c3e04918b/mpl_gui/_manage_backend.py#L87-L138 and contra @WeatherGod we have documented that mpl.use('module://XYZ) or mpl.rcParams['backend'] = 'module://XYZ' work back to 2008 (it came in via 032cc0c ) )
• module names (and the same space in a module) is mechanism that Python provides us which is guaranteed to be process unique (you can put the same foo.py file in more than one place in your PYTHONPATH and the first one Python finds wins but that is done by Python and hence Not Our Problem).
• backends (how ever we spell them) should be able to tell pyplot which GUI needs to be activated
• it is useful is some contexts for Matplotlib to be able to reach out and ask who wishes to be identified with what alias exist (an application make want to ask, for us to provide nice error messages, etc)

Hopefully everyone agrees these statements are true?

My proposal now is very similar to my one above:

• the mapping of backend -> input to ip.enable_gui moves to Matplotlib from IPython (and IPython will need to do some version gating in %matplotlib to keep back-compat)
• matplotlib picks up the ability to be passed a abkcend object and just use it straight away
• %matplotlib will now check the user namespace to see if there is an object with that name (and if it has it pass it through)
• Matplotlib becomes aware of an entry point to let packages register them selves with a name
• we consider m:// as a shorter alias for module://
• in pyplot.switch_backends we do:
• if the input is a backend object, just use it (with what ever jiggery-pokery we need to make sure that mpl.rcParams['backend'] is always a string
• if it is a string and it starts with module:// or m:// we drop the prefix and do (nested) imports
• if it is a string without a prefix
  • if it is an aliase for a backend that matplotlib ships in our code base we use our mapping (and if we merge new backends (thinking a head to mplcairo) we will expand this list asserting we own a subset of names)
  • (maybe) see if the name we have been given is importable, and if it is import it and try to use it, raise if it is importable, but not a backend
  • if it is an alias we do not know, then we look at an entry point. Not sure how to break ties if more than one package claims the alias, maybe raise with an error that gives the m:// form of both?
• finally raise if unkwons

Hopefully this gets everyone what they want?

• we do not force backend implementers into using the registry (so mplcairo can fully opt out of)
• the registry is available for implementers who want to use it (which ipympl and matplotlib_inline can use)
• we get as terse as the implementer wants alias for backends that opt-in
• we get very explicit guaranteed-to-always-work way to specifying a backend

I think there is some legitimate concern about "why do both?!", but we already support both and can not back out of either. If we are going to support aliases, we should make the pluggable.

I am having some hesitation about the implicit import via un-prefixed strings. There is a bit of @WeatherGod 's concern of surprise imports (but with aliases that also will import lots of things), but I am more concerned about the ambiguity of if you will get the module you named or some very differently named module which has claimed that name. I also think there is a non-zero risk of having an alias (because that is how you want to roll) and then some un-related package installs with that name and now we try (and fail) to import that

Can you clarify the following point?

%matplotlib will now check the user namespace to see if there is an object with that name (and if it has it pass it through)

I read that as use("foo") -> lookup globals()["foo"] in the caller's namespace, which is hopefully not what you want?

The rest looks good.

Can you clarify the following point?

If I recall correctly in the implementation of a magic it gets access to the input as strings as well as access to the ip.user_ns (I am 99% sure of this because there is no qt in the namespace, but %matplotlib qt works. This detail was to make sure you could do

In []: import my_backend as foo
in []: %matplotlib foo

This would be magic-side logic, I very much agree that mpl.use() should not reach out to the globals (and that would probably not even work as the users namespace is in ip.user_ns not in process globals).

But do you actually want

In []: import my_backend as foo
In []: %matplotlib foo

to work? I guess I don't have an overly strong opinion of the API on IPython's side, but that doesn't look too great to me?

I think we should have

matplotlib.use(<whatever>)  # the standard Matplotlib function; possibly implicit via matplotlibrc
%matplotlib  # no args, uses whatever Matplotlib has been configured to use

and as a convenience

%matplotlib foo

which maps to

matplotlib.use(resolve_alias("foo"))  # i.e. foo is understood as a string
%matplotlib

But do you actually want

I was thinking how to bring plt.switch_backends(<thing>) into as close parity as %matplotlib <thing>. If we grow the ability to pass objects directly to plt.switch_backend then it seems symmetric to also grow the ability to pass via the magic. It would be very easy to talk me out of including this.

My immediate reaction is that I don't really like it, but I don't feel that strongly about it either right now.

This detail was to make sure you could do

In []: import my_backend as foo
in []: %matplotlib foo

IPython magic is not python code. I don't think we should try and make it look like it. You also cannot do:

interactive = False
%matplotlib widget if interactive else inline

IPython magic is not python code. I don't think we should try and make it look like it.

OK, I'm sold on this position.

Actually, one point I did miss about the alias system: I guess this means that now use("foo") can indeed resolve in importing arbitrary python modules? (more precisely, whatever module that claims "foo" as an alias)
As stated above I don't think that's actually a problem (I argued very strongly that it isn't :-)), but perhaps still worth pointing out, especially as now the mapping between alias and module name is not obvious.

Since this Pull Request has not been updated in 60 days, it has been marked "inactive." This does not mean that it will be closed, though it may be moved to a "Draft" state. This helps maintainers prioritize their reviewing efforts. You can pick the PR back up anytime - please ping us if you need a review or guidance to move the PR forward! If you do not plan on continuing the work, please let us know so that we can either find someone to take the PR over, or close it.

Closing this as there will be a different implementation, see issue #27663.