Skip to content

Backend related issues with matplotlib 3.0.0rc1 #11844

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cgohlke opened this issue Aug 12, 2018 · 21 comments
Closed

Backend related issues with matplotlib 3.0.0rc1 #11844

cgohlke opened this issue Aug 12, 2018 · 21 comments
Milestone

Comments

@cgohlke
Copy link
Contributor

cgohlke commented Aug 12, 2018

I noticed three backend related issues with matplotlib 3.0.0rc1 on Windows:

  1. Setting backend = TkAgg in setup.cfg before building wheels, TkAgg does not get enabled as the default backend in the mpl-data/matplotlibrc file. That option has been removed in 56acb0f#diff-2eeaed663bd0d25b7e608891384b7298. I am aware this is (sort of) mentioned in the whats_new document. TkAgg has always been the default backend in the official binaries for Windows. IMO, this default has been more predictable, reliable (e.g. see issue 2), and faster loading than other backends.

  2. At least on my system with PyQt4 installed, the default backend is unusable:

>>> from matplotlib import pyplot
>>> pyplot.plot()
[]
>>> pyplot.show()
Traceback (most recent call last):
  File "X:\Python37\lib\site-packages\matplotlib\backends\backend_qt5agg.py", line 57, in paintEvent
    buf, buf.shape[1], buf.shape[0], QtGui.QImage.Format_RGBA8888)
AttributeError: type object 'QImage' has no attribute 'Format_RGBA8888' 

A window opens but no plot is rendered.

  1. I am unable to print the default backend:
>>> import matplotlib
>>> print(matplotlib.get_backend())
<object object at 0x000002928A3FE7A0> 

Matplotlib version

  • Operating system: Windows 10 Pro WS 1803
  • Matplotlib version: 3.0.0rc1
  • Matplotlib backend (print(matplotlib.get_backend())): <object object at 0x000001F814B0E7A0>
  • Python version: Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32
  • Jupyter version (if applicable):
  • Other libraries:

Installed from source matplotlib-3.0.0rc1.zip

@anntzer
Copy link
Contributor

anntzer commented Aug 12, 2018

For (2), support for Format_RGBA8888 was indeed only added in Qt 5.2. I have a fix coming for that.
(3) is essentially the point mentioned at #11600 (comment).

@tacaswell
Copy link
Member

We should put back in a way for packagers to have an opinion about the default backend (but also allow them not to).

@anntzer
Copy link
Contributor

anntzer commented Aug 13, 2018

Basically what we had before, but defaulting to the fallback mechanism if setup.cfg doesn't set anything special?
Alternatively (and preferably), I'd suggest them to just apply whatever patch to matplotlibrc.template they want...

@tacaswell
Copy link
Member

Basically what we had before, but defaulting to the fallback mechanism if setup.cfg doesn't set anything special?

Yes

Alternatively (and preferably), I'd suggest them to just apply whatever patch to matplotlibrc.template they want...

That seems more brittle to me.

@tacaswell tacaswell added this to the v3.0 milestone Aug 13, 2018
@matthew-brett
Copy link
Contributor

I am guessing the following is relevant, but please correct me if I am wrong. I'm getting new errors on the mpl wheel builds:

  /venv/lib/python3.6/site-packages/matplotlib/testing/__init__.py:36: UserWarning: Could not set locale to English/United States. Some date-related tests may failTraceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/venv/lib/python3.6/site-packages/matplotlib/pyplot.py", line 2410, in <module>
    switch_backend(rcParams["backend"])
  File "/venv/lib/python3.6/site-packages/matplotlib/pyplot.py", line 260, in switch_backend
    newbackend, required_framework, current_framework))
ImportError: Cannot load backend 'TkAgg' which requires the 'tk' interactive framework, as 'headless' is currently running

https://travis-ci.org/MacPython/matplotlib-wheels/jobs/415473608

@anntzer
Copy link
Contributor

anntzer commented Aug 13, 2018

@tacaswell I'm fine with keeping the configurability for now (given that it basically is already present, and just needs to be adapted for #11600), but I'm not convinced that making it easier for downstream packagers to distribute various slightly different things as "matplotlib" is actually that great (for example, what, exactly, is the use case we're talking about here?)

@matthew-brett Likely related, but it's unclear from the log which actual test is failing (and causing the build to fail)?

@cgohlke
Copy link
Contributor Author

cgohlke commented Aug 13, 2018

what, exactly, is the use case we're talking about here?

The official Windows binaries have shipped with TkAgg as the default backend for almost a decade (at least since https://sourceforge.net/p/matplotlib/mailman/message/22753475/). I don't see any good reason to change that but it is of course up to the mpl maintainers.

On Anaconda, the default backends du jour are qt5agg (Windows), macosx (Mac), and tkagg (Linux). I assume this is for reasons of QA or support (?).

@anntzer
Copy link
Contributor

anntzer commented Aug 13, 2018

As stated above, I'd rather have every "matplotlib" out there in the wild do "the same thing". That's also sort of the point of #11600 (or at least one of them)...

@jklymak
Copy link
Member

jklymak commented Aug 13, 2018

I don't see any reason for the mac backend to not be Qt5Agg; I use anaconda, and that is the backend I use, and I know @efiring does as well.

@matthew-brett
Copy link
Contributor

That could make sense if everyone was using Anaconda, but that really isn't the case. I effectively only use the macOS backend myself.

@matthew-brett
Copy link
Contributor

@anntzer - I have no more information than you do about which test is causing the failure, unfortunately. I see the tests only fail on Linux. I'm guessing that it would be possible to replicate on a Linux system without the Tk libraries installed.

@jklymak
Copy link
Member

jklymak commented Aug 13, 2018

@matthew-brett Is Qt5 hard to install if you don't use anaconda? Sorry, I haven't tried in years...

@matthew-brett
Copy link
Contributor

I haven't tried recently - partly because it's such a huge install, and I often use virtualenvs. It looks like there's a wheel for PyQt5, but I don't know whether it can be relied on in general. I believe it's relatively recent.

My impression was that John Hunter, at least, thought of the matplotlib idea as supporting multiple backends to an acceptable degree of usability, in order that people can use their backend of choice, rather than having to switch their plotting library, in order to use their preferred backend.

@efiring
Copy link
Member

efiring commented Aug 13, 2018

That's correct; but not all backends that have been included at some point in mpl's history have survived.
The macosx backend seems to need more maintenance than we have been giving it since @mdehoon left. I hadn't tried it for a long time, so just now I brought it up in ipython with "%matplotlib osx", and made a trivial plot. The first thing I notice is that the cursor readout works only if the trackpad is held down with one finger while moving the cursor with another. I don't think this used to be the case, and it is not the way other backends work. I don't know how many other anomalies like this there are, but this alone is a big one.

Qt is a large dependency. The conda-forge pkg file is 80 MB:
-rw-rw-r-- 1 efiring efiring 80543764 Jul 7 10:53 qt-5.9.5-h02808f3_0.tar.bz2

I certainly understand the rationale for having macosx--and also the rationale for dropping it, unless it can be more actively maintained.

@jklymak
Copy link
Member

jklymak commented Aug 14, 2018

I think there are two parallel discussions: 1) is backend-x worth keeping around? and 2) should matplotlib have a default list of backend to try that all distributions start with as their default, but users can still change either at run time or using and rcParam. I think @anntzer is arguing for the latter, and my comment was only meant to indicate that Qt5Agg as a default for anaconda/macOS isn't a terrible choice from a functionality point of view.

I'm pretty sure that in a venv that the user didn't want to drag around Qt5 for, the fallback could still be to macosx, and again, they can still force that with their rcParams.

@anntzer
Copy link
Contributor

anntzer commented Aug 14, 2018

Indeed, I am absolutely not arguing in favor of removing the OSX backend (it may need some maintenance, but that's an irrelevant point). The point is only to have a uniform list of backends that gets tried in that order until one succeeds (and because macosx is the first on that list, it will remain the default backend on OSX -- unless we decide to change that). In fact, note that this is actually the current behavior if you build matplotlib yourself locally -- except that the "try-one-backend-at-a-time" is only performed once, at build time, rather than at runtime (and the actual order of tried backends has been kept the same in #11600). Obviously, with the rise of pre-built matplotlibs (i.e., linux distros and wheels), that cannot work (because the available backends is not the same on the build machine and on the end-user machine); in my view that's likely (one of) the reasons why the configurability of the default has been put in.

@anntzer
Copy link
Contributor

anntzer commented Aug 14, 2018

re @tacaswell in #11852, reproduced here for clarity:

The simplest fix may be to switch the sentinel to the string 'auto'? That way it can go in the .matplotlibrc and returns the expected type from get_backend()

This creates another set of issues: Do you except people to be able to set "auto" and get fallback? Or {fallback or not fallback} depending on whether pyplot has already been imported? What GUI framework integration is IPython supposed to setup when it gets "auto" as backend? Should IPython import pyplot itself and trigger fallback resolution? Note that that would likely mean you'd need a new IPython release.
Conversely, if you just make get_backend() trigger fallback, that'll likely play more nicely with IPython.

I would prefer that if we keep the non-string sentinel that the backend resolution logic moves up to __init__ rather than adding another buried pyplot import.

Fundamentally, use() is a pyplot API; it only lives in the toplevel because the implementation before #11600 forced its use before pyplot was imported. Until this API gets changed (if ever), I don't see the problem with importing pyplot from use().

Again, I think that there used to be some reluctance in importing pyplot because that was, so far, an irreversible step in setting the backend. But this is no longer the case (the backend is locked in only when an actual pyplot-managed window is opened), so we shouldn't feel bad about importing pyplot.

@tacaswell
Copy link
Member

This creates another set of issues: Do you except people to be able to set "auto" and get fallback? Or {fallback or not fallback} depending on whether pyplot has already been imported?

While confusing, this seems better than the type instability we currently have.

What GUI framework integration is IPython supposed to setup when it gets "auto" as backend? Should IPython import pyplot itself and trigger fallback resolution? Note that that would likely mean you'd need a new IPython release.
Conversely, if you just make get_backend() trigger fallback, that'll likely play more nicely with IPython.

IPython already imports pyplot when you use the %matplotlib magic, but it is not clear on the order. Getting that to work may be the biggest fly in this ointment...

@anntzer
Copy link
Contributor

anntzer commented Aug 20, 2018

While confusing, this seems better than the type instability we currently have.

Well, the point of making rcParams.__getitem__("backend") perform backend resolution is exactly to avoid exposing the type instability.

@tacaswell
Copy link
Member

3 has been addressed, 2 is just waiting on CI, 1 still needs to be done.

@tacaswell
Copy link
Member

Closed by #11962

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants