Skip to content

backend switching. #11581

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

backend switching. #11581

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/api/next_api_changes/2018-02-15-AL-deprecations.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ The following classes, methods, functions, and attributes are deprecated:
- ``backend_ps.get_bbox``,
- ``backend_qt5.error_msg_qt``, ``backend_qt5.exception_handler``,
- ``backend_wx.FigureCanvasWx.macros``,
- ``backends.pylab_setup``,
- ``cbook.GetRealpathAndStat``, ``cbook.Locked``,
- ``cbook.is_numlike`` (use ``isinstance(..., numbers.Number)`` instead),
``cbook.listFiles``, ``cbook.unicode_safe``,
Expand Down
22 changes: 22 additions & 0 deletions doc/api/next_api_changes/2018-06-27-AL.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,28 @@
Changes to backend loading
``````````````````````````

It is now possible to set ``rcParams["backend"]`` to a *list* of candidate
backends.

If `.pyplot` has already been imported, Matplotlib will try to load each
Copy link
Member

@jklymak jklymak Jul 6, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By ".pyplot has already been imported" do you mean matplotlib.use() hasn't been called? Does pyplot fundamentally have anything to do with this, other than its where 99% of us call matplotlib.use() implicitly?

candidate backend in the given order until one of them can be loaded
successfully. ``rcParams["backend"]`` will then be set to the value of the
successfully loaded backend. (If `.pyplot` has already been imported and
``rcParams["backend"]`` is set to a single value, then the backend will
likewise be updated.)

If `.pyplot` has not been imported yet, then ``rcParams["backend"]`` will
maintain the value as a list, and the loading attempt will occur when `.pyplot`
is imported. If you rely on ``rcParams["backend"]`` (or its synonym,
``matplotlib.get_backend()`` always being a string, import `.pyplot` to trigger
backend resolution.

`.pyplot.switch_backends` (but not `matplotlib.use`) have likewise gained the
ability to accept a list of candidate backends.

In order to support the above features, the additional following changes were
made:

Failure to load backend modules (``macosx`` on non-framework builds and
``gtk3`` when running headless) now raises `ImportError` (instead of
`RuntimeError` and `TypeError`, respectively.
Expand Down
69 changes: 17 additions & 52 deletions lib/matplotlib/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1317,6 +1317,7 @@ def __exit__(self, exc_type, exc_value, exc_tb):
dict.update(rcParams, self._orig)


# FIXME: Remove.
_use_error_msg = """
This call to matplotlib.use() has no effect because the backend has already
been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
Expand All @@ -1329,62 +1330,26 @@ def __exit__(self, exc_type, exc_value, exc_tb):

def use(arg, warn=True, force=False):
"""
Set the matplotlib backend to one of the known backends.
Set the Matplotlib backend.

The argument is case-insensitive. *warn* specifies whether a
warning should be issued if a backend has already been set up.
*force* is an **experimental** flag that tells matplotlib to
attempt to initialize a new backend by reloading the backend
module.
The argument is case-insensitive. Switching to an interactive backend is
only safe if no event loop for another interactive backend has started.
Switching to and from non-interactive backends is safe.

.. note::

This function must be called *before* importing pyplot for
the first time; or, if you are not using pyplot, it must be called
before importing matplotlib.backends. If warn is True, a warning
is issued if you try and call this after pylab or pyplot have been
loaded. In certain black magic use cases, e.g.
:func:`pyplot.switch_backend`, we are doing the reloading necessary to
make the backend switch work (in some cases, e.g., pure image
backends) so one can set warn=False to suppress the warnings.

To find out which backend is currently set, see
:func:`matplotlib.get_backend`.
To find out which backend is currently set, see `matplotlib.get_backend`.

Parameters
----------
arg : str
The name of the backend to use.
"""
# Lets determine the proper backend name first
if arg.startswith('module://'):
name = arg
else:
# Lowercase only non-module backend names (modules are case-sensitive)
arg = arg.lower()
name = validate_backend(arg)

# Check if we've already set up a backend
if 'matplotlib.backends' in sys.modules:
# Warn only if called with a different name
if (rcParams['backend'] != name) and warn:
import matplotlib.backends
warnings.warn(
_use_error_msg.format(
backend=rcParams['backend'],
tb=matplotlib.backends._backend_loading_tb),
stacklevel=2)

# Unless we've been told to force it, just return
if not force:
return
need_reload = True
else:
need_reload = False

# Store the backend name
rcParams['backend'] = name

# If needed we reload here because a lot of setup code is triggered on
# module import. See backends/__init__.py for more detail.
if need_reload:
importlib.reload(sys.modules['matplotlib.backends'])
if not isinstance(arg, str):
# We want to keep 'use(...); rcdefaults()' working, which means that
# use(...) needs to force the default backend, and thus be a single
# string.
raise TypeError("matplotlib.use takes a single string as argument")
rcParams["backend"] = \
rcParamsDefault["backend"] = rcParamsOrig["backend"] = arg


if os.environ.get('MPLBACKEND'):
Expand Down
7 changes: 6 additions & 1 deletion lib/matplotlib/backend_bases.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3210,6 +3210,10 @@ class _Backend(object):
# class FooBackend(_Backend):
# # override the attributes and methods documented below.

# Set to one of {"qt5", "qt4", "gtk3", "wx", "tk", "macosx"} if an
# interactive framework is required, or None otherwise.
required_interactive_framework = None

# `backend_version` may be overridden by the subclass.
backend_version = "unknown"

Expand Down Expand Up @@ -3292,7 +3296,8 @@ def show(cls, block=None):

@staticmethod
def export(cls):
for name in ["backend_version",
for name in ["required_interactive_framework",
"backend_version",
"FigureCanvas",
"FigureManager",
"new_figure_manager",
Expand Down
3 changes: 3 additions & 0 deletions lib/matplotlib/backends/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,13 @@
import traceback

import matplotlib
from matplotlib import cbook
from matplotlib.backend_bases import _Backend

_log = logging.getLogger(__name__)

backend = matplotlib.get_backend()
# FIXME: Remove.
_backend_loading_tb = "".join(
line for line in traceback.format_stack()
# Filter out line noise from importlib line.
Expand Down Expand Up @@ -64,6 +66,7 @@ def _get_running_interactive_framework():
return None


@cbook.deprecated("3.0")
def pylab_setup(name=None):
"""
Return new_figure_manager, draw_if_interactive and show for pyplot.
Expand Down
72 changes: 59 additions & 13 deletions lib/matplotlib/pyplot.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@
The object-oriented API is recommended for more complex plots.
"""

import importlib
import inspect
import logging
from numbers import Number
import re
import sys
Expand Down Expand Up @@ -67,10 +69,13 @@
MaxNLocator
from matplotlib.backends import pylab_setup

_log = logging.getLogger(__name__)


## Backend detection ##


# FIXME: Deprecate.
def _backend_selection():
"""
If rcParams['backend_fallback'] is true, check to see if the
Expand Down Expand Up @@ -110,8 +115,6 @@ def _backend_selection():
## Global ##


_backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()

_IP_REGISTERED = None
_INSTALL_FIG_OBSERVER = False

Expand Down Expand Up @@ -213,21 +216,61 @@ def findobj(o=None, match=None, include_self=True):

def switch_backend(newbackend):
"""
Switch the default backend. This feature is **experimental**, and
is only expected to work switching to an image backend. e.g., if
you have a bunch of PostScript scripts that you want to run from
an interactive ipython session, you may want to switch to the PS
backend before running them to avoid having a bunch of GUI windows
popup. If you try to interactively switch from one GUI backend to
another, you will explode.
Close all open figures and set the Matplotlib backend.

The argument is case-insensitive. Switching to an interactive backend is
possible only if no event loop for another interactive backend has started.
Switching to and from non-interactive backends is always possible.

Calling this command will close all open windows.
Parameters
----------
newbackend : str or List[str]
The name of the backend to use. If a list of backends, they will be
tried in order until one successfully loads.
"""
close('all')

if not isinstance(newbackend, str):
for candidate in newbackend:
try:
_log.info("Trying to load backend %s.", candidate)
return switch_backend(candidate)
except ImportError as exc:
_log.info("Loading backend %s failed: %s", candidate, exc)
else:
raise ValueError("No suitable backend among {}".format(newbackend))

backend_name = (
newbackend[9:] if newbackend.startswith("module://")
else "matplotlib.backends.backend_{}".format(newbackend.lower()))

backend_mod = importlib.import_module(backend_name)
Backend = type(
"Backend", (matplotlib.backends._Backend,), vars(backend_mod))
_log.info("Loaded backend %s version %s.",
newbackend, Backend.backend_version)

required_framework = Backend.required_interactive_framework
current_framework = \
matplotlib.backends._get_running_interactive_framework()
if (current_framework and required_framework
and current_framework != required_framework):
raise ImportError(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am worried that by this point we are already broken (due to importing the underlying GUI wrappers which may over-write each other's PyOS_EventHook), but I don't see any way around that short of delaying all of the imports (which I am not sure we can do).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyOS_InputHook should (hopefully) not be overwritten before the toolkit event loop actually starts; e.g. for PyQt:

from PyQt5.QtWidgets import *
print(input("foo? "))
app = QApplication([])
print(input("foo? "))

even after the QApplication is created, input works fine; overwriting likely only occurs in app.exec_().

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if you the import tk will the Qt windows still be responsive while waiting for user input?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes (this can easily be checked by, well, doing it). I'd guess tk also only sets up the input hook when actually starting the event loop.

"Cannot load backend {!r} which requires the {!r} interactive "
"framework, as {!r} is currently running".format(
newbackend, required_framework, current_framework))

rcParams["backend"] = newbackend

global _backend_mod, new_figure_manager, draw_if_interactive, _show
matplotlib.use(newbackend, warn=False, force=True)
from matplotlib.backends import pylab_setup
_backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
_backend_mod = backend_mod
new_figure_manager = Backend.new_figure_manager
draw_if_interactive = Backend.draw_if_interactive
_show = Backend.show

# Need to keep a global reference to the backend for compatibility reasons.
# See https://github.com/matplotlib/matplotlib/issues/6092
matplotlib.backends.backend = newbackend


def show(*args, **kw):
Expand Down Expand Up @@ -2358,6 +2401,9 @@ def _autogen_docstring(base):
# to determine if they should trigger a draw.
install_repl_displayhook()

# Set up the backend.
switch_backend(rcParams["backend"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step locks the backend in, @WeatherGod wants

import matplotlib
matplotlib.use('qt5agg')
import matplotlib.pyplot
matplotlib.use('tkagg')  # this does not warn or fail
plt.figure()  # this makes a tk agg figure and actually does the backend import etc.

Copy link
Contributor Author

@anntzer anntzer Jul 8, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it doesn't, because the QApplication isn't created until a figure is created. Indeed, your snippet works as expected.
Sure, the snippet would fail with ImportError if PyQt5 is not installed, but that seems to be asking a bit too much (just wrap it in a try... except yourself); in fact I think a fast fail is less confusing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am apparently confused about the details of various hooks get installed or not.... 🐑

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general nothing should happen until you create a widget; we then start the loop lazily.



################# REMAINING CONTENT GENERATED BY boilerplate.py ##############

Expand Down
43 changes: 37 additions & 6 deletions lib/matplotlib/rcsetup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,13 @@
parameter set listed here should also be visited to the
:file:`matplotlibrc.template` in matplotlib's root source directory.
"""

from collections import Iterable, Mapping
from functools import reduce
import operator
import os
import re
import sys

from matplotlib import cbook
from matplotlib.cbook import ls_mapper
Expand Down Expand Up @@ -245,10 +247,35 @@ def validate_fonttype(s):


def validate_backend(s):
if s.startswith('module://'):
return s
candidates = _listify_validator(
lambda s:
s if s.startswith("module://")
else ValidateInStrings('backend', all_backends, ignorecase=True)(s))(s)
pyplot = sys.modules.get("matplotlib.pyplot")
if len(candidates) == 1:
backend, = candidates
if pyplot:
# This import needs to be delayed (below too) because it is not
# available at first import.
from matplotlib import rcParams
# Don't recurse.
old_backend = rcParams["backend"]
if old_backend == backend:
return backend
dict.__setitem__(rcParams, "backend", backend)
try:
pyplot.switch_backend(backend)
except Exception:
dict.__setitem__(rcParams, "backend", old_backend)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be done like this to make sure we un-wind any changes that `switch_backend_ makes to the rcparams?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because switch_backend will set rcParams["backend"] too (to keep things consistent), but we don't want this to recurse ad infinitum (so we pre-set the entry -- but undo that if at the end of the day the backend switch was unsuccessful).

Really it's just a complication because of the existence of two entry points (setting rcParams["backend"] and switch_backend) to the same info; it would even be simpler if switch_backend itself was inlined into the validator (but then you have funky action at distance with the proper initialization of pyplot happening in some other module...).

raise
return backend
else:
return _validate_standard_backends(s)
if pyplot:
from matplotlib import rcParams
pyplot.switch_backend(candidates) # Actually resolves the backend.
return rcParams["backend"]
else:
return candidates


def validate_qt4(s):
Expand Down Expand Up @@ -965,9 +992,13 @@ def _validate_linestyle(ls):

# a map from key -> value, converter
defaultParams = {
'backend': ['Agg', validate_backend], # agg is certainly
# present
'backend_fallback': [True, validate_bool], # agg is certainly present
'backend': [["macosx",
"qt5agg", "qt4agg",
"gtk3agg", "gtk3cairo",
"tkagg",
"wxagg",
"agg", "cairo"], validate_backend],
'backend_fallback': [True, validate_bool],
'backend.qt4': [None, validate_qt4],
'backend.qt5': [None, validate_qt5],
'webagg.port': [8988, validate_int],
Expand Down
Loading