This bisects to #12637, and is essentially due to the fact that we now initialize ipython/matplotlib support when the first canvas is created (here, by plt.figure()), that during initialization, ipython calls switch_backend, that switch_backend starts by calling close("all"), and that NXPlotView() is registered with pyplot and thus gets closed at that point.

I think we can actually remove the close("all") (well, modulo backcompat, yada yada)? If there are conflicting event loops, closing figures won't help, if there aren't (like in this case), we don't need to close them.

In the meantime you can probably get away with creating a figure and immediately closing it -- we only initialize ipython/matplotlib support once.

Thanks for the information. I am subclassing FigureCanvasQT, so I would prefer to preempt the close("all") call, perhaps by callingswitch_backend myself for the first instance, but I can't see how it gets called during the first plt.figure() call. I'll look again tomorrow, but if you have any tips, I would appreciate them. I wondered if it was hidden in the switch_backend_decorator, but I don't see that being used.

switch_backend is called via ip.enable_matplotlib(), which is called in FigureCanvasBase._fix_ipython_backend2gui, which is called in the FigureCanvasBase constructor.

I think removing that close("all") might be the only solution for me. I have tried endless permutations of preemptively setting mpl.rcParamsOrig('backend'), calling pt.enable_matplotlib or pt.activate_matplotlib, adding a dummy _fix_ipython_backend2gui to my FigureCanvasQT subclass, even defining my own subclass of _BackendQT5 using the _Backend.export decorator (which produces unhelpful side effects) , but none of them stop IPython from calling switch_backend, whose first line is close("all"). I need to make a new release of NeXpy in the next couple of days, but I am going to have to make Matplotlib v3.0.3 the highest allowed version, unless anyone has other tips for me to try.

It looks like the following workaround does suffice: add a call to FigureCanvas(Figure()) at the toplevel of plotview.py (note that because the figure is not registered with pyplot, it will be immediately garbage collected anyways, but that'll arrange to call _fix_ipython_backend2gui properly).

(I still do think that we should not call close("all"), but that may take longer to change...)

Thanks for the suggestion, which I think would work. In the past few minutes, I have just found an alternative solution. I just have to monkey-patch the IPython InteractiveShell class to do nothing when there is a call to enable_matplotlib. Everything else is already initialized so the call is redundant. It also seems to work.

I haven't thought about this issue nearly as long as you, but I get the impression that the ultimate solution would have to come from IPython allowing alternative backends to be registered in some well-documented way.

The way we are registering backends with IPython definitely borders API abuse, sorry for breaking things on your side. On the other hand, I would rather have as much of that code as possible living on Matplotlib's side, as cross-project coordination is a much harder problem...

I remilestone to 3.2 as we have a workaround now, although I do think the API needs to be revisited on our side.

I think the simplest short-term fix would be to add an optional keyword argument to the FigureCanvasBase to allow instances to skip the call to _fix_ipython_backend2gui if it's unnecessary. If you are loath to change call signatures, the same thing could be achieved by adding a private class variable (_fix_ipython=True) that a subclass could override.

That doesn't seem much better than just documenting "call FigureCanvas(Figure()) early"?

I am a little rushed at the moment, so I may be wrong about this, but I think the problem with calling FigureCanvas(Figure()) in NeXpy is that the IPython shell doesn't exist when I initialize the plotting windows, so the call to IPython.get_ipython() would fail. I could probably reconfigure the initialization process I would prefer not to refactor my code if possible, and I think Matplotlib should allow for situations where the IPython fix is completely unnecessary.

As it is written right now, FigureCanvas(Figure()) will call _fix_ipython_backend2gui regardless of whether IPython was already initialized, and _fix_ipython_backend2gui will not be called a second time (due to the functools.lru_cache() decorator) even if IPython is later initialized (I didn't think about the embedding case at all when I wrote this). So the fix works (... at least for me) even if you call FigureCanvas(Figure()) at the toplevel of the module.

I must admit that I had been puzzled about how subsequent calls were suppressed. I didn't know what the functools.lru_cache() decorator did. I am happy to leave this to real Matplotlib experts such as you now that I have my own solution and you are aware of the issue. If you reference this issue when any relevant changes are implemented, I should be able to keep my own code compatible. This isn't the first bit of monkey-patching I've had to do - it's one of the hazards of embedding other packages.

I propose #14471 as a fix which avoids closing the figures if the backend switch is a no-op.

@anntzer Documenting that import / instantiation order seems a bit brittle.

@rayosborn I would much rather not have you monkey patching (some what selfishly as I suspect I'll be on both sides of that interaction sooner or later ;) ).

I agree it's brittle. The problem of #14471 is that I could possibly imaging some people are actually (implicitly?) relying on switch_backend() to call close("all"), but I'll trust your judgment that the backcompat break is fine.

Hmm, I may have miss-understood the source of the behavior change between 3.0 and 3.1. I thought it was a change to switch_backend but reading this thread more carefully (and checking blame), the change is actually the call to ip.enable_matplotlib().

Could we put a guard in _fix_ipython_backend2gui to not make that call if the user has already started using pyplot? That also has problems of more implicit magic....

@rayosborn do you actually want your figures to be known to pyplot?

(an important point is that ip.enable_matplotlib calls switch_backend)

@tacaswell the NeXpy figures use a subclass of FigureCanvasQT, which ensures that pyplot at least knows they exist, e.g., they are included in plt.get_fignums, etc. This still works fine. The problem only arises if a user uses pyplot to create a figure from the IPython shell (plt.figure()). This creates a regular plotting window using the Matplotlib QT backend with an incremented figure number. That allows the Matplotlib power user to create their own plots without the restrictions that NeXpy has (e.g., with lots of subplots), so I want to retain that option and allow the two types of plotting windows to co-exist.

My monkey patch is just to disable the redundant IPython call to enable_matplotlib by assigning the IPython shell function to a dummy function that does nothing. I think it's fairly safe because it's a function that should never be called within NeXpy, but I could probably remove it again if Matplotlib manages to avoid calling switch_backend.

which ensures that pyplot at least knows they exist, e.g., they are included in plt.get_fignums, etc

But do you want this behavior?

https://github.com/nexpy/nexpy/blob/8e83badf6d38092cfc6abaf5d53e1a162db8830c/src/nexpy/gui/plotview.py#L289
https://github.com/nexpy/nexpy/blob/8e83badf6d38092cfc6abaf5d53e1a162db8830c/src/nexpy/gui/plotview.py#L564

are the lines that let pyplot know your figures exist. If you dropped those lines the nexpy plots and the plt.xxx plots would live in essentially independent namespaces and would not conflict (ie nexpy windows won't be in plt.get_fignums and will be immune to plt.close('all'))

That's an interesting point. I'll think about that. It might be worth cutting the cord.

@tacaswell, thanks for the suggestion. As you have seen, I already merged a NeXpy PR where I separate the NXPlotView and pyplot figures, and I think it's a big improvement, even without this issue, so it's had a very useful outcome.