Skip to content

[Bug]: Embedded Python uncatchable hard exit on closing last matplotlib window. #27147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
prniii opened this issue Oct 19, 2023 · 13 comments
Open

Comments

@prniii
Copy link

prniii commented Oct 19, 2023

Bug summary

On macOS, default backend (Cocoa?), with Python embedded into an application, closing the last open Matplotlib window causes an uncatchable exit. It does not throw a system exit exception to call Python's system exit, but rather a C level exit which causes the entire application to close. This is fine if you are running Matplotlib from an interactive console, it is not fine when Python is embedded as a scripting language within an application.

Code for reproduction

# Embed Python into a minimal application, run application 
# from the embedded python scripting, run the script.
# Such as running through PyRun_SimpleString("code");  (not an interactive shell)

import matplotlib.pyplot as plt
plt.clf()         # make sure we are starting with a clean plot
plt.plot([1, 2, 3, 4])
plt.ylabel('some numbers')
plt.show()

# close the window - application exits in an uncatchable manner.

Actual outcome

Application shutdown that is not catchable.

Expected outcome

Window closes and script stops running and control is returned to interpreter, but Application with Python interpreter continues running. Backend returns to a state waiting for another plot / window to be opened.

Additional information

Use Matplotlib from an application that embeds Python as a scripting language. Close last Matplotlib window. Matplotlib forces application to close ungracefully, no way to catch close, no way to save / close opened files no way for an orderly shutdown and a C level exit is called.

Matplotlib always forces exit for every version tried.
Why: A call to exit() or exit_() at the C level of the code, not a Python SystemExit.
Fix: Don't terminate the application with an uncatchable exit.

Operating system

macOS

Matplotlib Version

3.8.0 and prior

Matplotlib Backend

MacOSX

Python version

3.11.5 ( and earlier all the way back to 3.5)

Jupyter version

No response

Installation

pip

@tacaswell
Copy link
Member

Do you have the same problem with any other GUI backed?

@prniii
Copy link
Author

prniii commented Oct 19, 2023 via email

@greglucas
Copy link
Contributor

Do you have a minimal example application you can give us to try and run this locally with?

# Embed Python into a minimal application, run application 
# from the embedded python scripting, run the script.
# Such as running through PyRun_SimpleString("code");  (not an interactive shell)

@prniii
Copy link
Author

prniii commented Oct 19, 2023 via email

@prniii
Copy link
Author

prniii commented Oct 20, 2023

matplotlib-27147.tgz

Xcode Objective C++ project that creates a main window GUI with a single pushbutton. Push the button, and it runs the Python code that runs matplotlib which raises a window. Close the matplotlib window and the app crashes. Python Code run is within AppController.mm file. Inside is a #define MACOSX_BACKEND 1 to choose between 2 python scripts to submit to the embedded Python. The first runs Matplotlib without specifying a backend so it gets the 'macosx' backend. setting it to 0 runs the same code except the first thing it does is set the backend to 'PyAgg'. The 'MacOSX' backend crashes, while the 'PyAgg' backend lets you. close the window, and click the button, close the window, click the button,..., repeat, ...

My current environment is macOS Ventura 13.6 Apple Silicon, Xcode 15, Python 3.11.5
This has been as issue for quite some time and crashed / exited on my prior Intel based MacBook Pro as well.

( use PyQt6 on Apple Silicon Mac since PyQt5 wouldn't install )
Assumptions -
standard Python 3.11.x is installed in default location by Python.org installer
installed into /Library/Frameworks/Python.framework - This app built with ( 3.11.5 )
packages installed with python3.11 -m pip install PyQt6 numpy matplotlib

App creates a Window with a button click me. The button's action runs the Python code
via PyRun_SimpleString( my_code );

@prniii
Copy link
Author

prniii commented Oct 20, 2023

I set the build's minimum required OS to 11.0 so it should build an run on macOS 11.0 or newer (Intel or Apple Silicon)

@prniii
Copy link
Author

prniii commented Oct 20, 2023

Even simpler example, single file can be compiled at the command line:
File: pyEmbed-matplotlib.mm

// Python embedding example - Matplotlib bug #27147
// Even simpler example
// compile with:
// clang++ -std=c++14 -o pyEmbed pyEmbed-matplotlib.mm `python3-config --includes` `python3-config --ldflags` -lpython3.11 -framework Cocoa
#include <stdio.h>
#include<Python.h>
#import <Cocoa/Cocoa.h>

void exec_pycode(const char*code)
{
	PyRun_SimpleString(code);
	//	Py_Finalize();
}

#define MACOSX_BACKEND 1

@interface myBtn : NSButton {
}
- (id)initWithFrame:(CGRect)frame;
@end

@implementation myBtn

- (id)initWithFrame:(CGRect)frame
{
  self = [super initWithFrame:frame];
  
  return self;
}

- (void)myAction
{
    static char crashOnClose[] = R"(
import matplotlib.pyplot as plt
import numpy as np
t = np.arange(0.0, 2.0, 0.01)
s = 1 + np.sin(2*np.pi*t)
plt.plot(t, s)
plt.xlabel('time (s)')
plt.ylabel('voltage (mV)')
plt.title('About as simple as it gets, folks')
plt.grid(True)
plt.show()
)";
    static char worksOnClose[] = R"(
import matplotlib
matplotlib.use("QtAgg")
import matplotlib.pyplot as plt
import numpy as np
t = np.arange(0.0, 2.0, 0.01)
s = 1 + np.sin(2*np.pi*t)
plt.plot(t, s)
plt.xlabel('time (s)')
plt.ylabel('voltage (mV)')
plt.title('About as simple as it gets, folks')
plt.grid(True)
plt.show()
)";


#if MACOSX_BACKEND
    PyRun_SimpleString(crashOnClose);
#else  // QtAgg backend
    PyRun_SimpleString(worksOnClose);  // you can repeatedly close window and re-click button
#endif // MACOSX_BACKEND define
    
}
@end

int main() {
  Py_Initialize();
  @autoreleasepool {
    NSApplication * app = [NSApplication sharedApplication];
    [NSApp setActivationPolicy:NSApplicationActivationPolicyRegular];
    id menubar = [[NSMenu new] autorelease];
    id appMenuItem = [[NSMenuItem new] autorelease];
    [menubar addItem:appMenuItem];
    [NSApp setMainMenu:menubar];
    id appMenu = [[NSMenu new] autorelease];
    id appName = [[NSProcessInfo processInfo] processName];
    id quitTitle = [@"Quit " stringByAppendingString:appName];
    id quitMenuItem = [[NSMenuItem alloc] initWithTitle:quitTitle
												 action:@selector(terminate:) keyEquivalent:@"q"]; //  autorelease];
    [appMenu addItem:quitMenuItem];
    [appMenuItem setSubmenu:appMenu];
    NSWindow * window = [[[NSWindow alloc] initWithContentRect:NSMakeRect(0, 0, 200, 200)
											styleMask:NSTitledWindowMask backing:NSBackingStoreBuffered defer:NO]
	 			  autorelease];
    [window cascadeTopLeftFromPoint:NSMakePoint(20,20)];
    [window setTitle:appName];
	myBtn *button = [[myBtn alloc] initWithFrame:NSMakeRect(10,60, 80, 60)];
	[button setTitle:@"Click me"];
	[[window contentView] addSubview:button];
	[button setTarget: button];
	[button setAction:@selector(myAction)];
	[button release];
	
    [window makeKeyAndOrderFront:nil];
    [NSApp activateIgnoringOtherApps:YES];
    [NSApp run];
  }
    return 0;
}

@prniii
Copy link
Author

prniii commented Oct 20, 2023

@ksunden
Copy link
Member

ksunden commented Oct 20, 2023

When I run when the app first opens I get:

ERROR: Setting <View: 0x13f87c990> as the first responder for window <Window: 0x117ad81c0>, but it is in a different window ()! This would eventually crash when the view is freed. The first responder will be set to nil.

(Along with what appears to be a stack trace)

We have this message here:

matplotlib/src/_macosx.m

Lines 529 to 530 in fcd5bb1

[window makeFirstResponder: view];
[[window contentView] addSubview: view];

I am able to make that warning go away by swapping the order of those two lines (so that the view is attached to the window before it is set as the first responder.

However, that does not actually change any behavior.

I am also seeing two more errors:

Unable to open mach-O at path: default.metallib  Error:2


No error handler for XPC error: Connection invalid

The first shows up when the window opens, the second when it closes.

Neither matplotlib, nor the example app appear to make any reference to metallib, so not sure where that is being triggered.

Some quick searching seems to indicate that the second is most often noise and not a real problem.

Neither show up in the qtagg case.

Now, I honestly don't know how to do it with the macosx backend (or even 100% if it is possible), but when I think embedded, I generally think more of actually displaying as a panel inside of another app, rather than opening a new window with default behavior. We have several examples of this in our docs (just not with macos, and assuming that you are writing the app in python): https://matplotlib.org/stable/gallery/user_interfaces/index.html

In these cases generally we do not use the pyplot interface (at all, not even for plt.subplots) and adding the canvas and toolbar directly into the gui framework's layout system.

These objects are defined in objective C code for the macos backend, but I'm not totally clear on how you would instantiate them and use them from objective C code. (we do not provide header files or anticipate you using the objC api directly..., so I think it would have to come through python objects, and not sure how that would work)

@prniii
Copy link
Author

prniii commented Oct 20, 2023 via email

@ksunden
Copy link
Member

ksunden commented Oct 21, 2023

When you say "PyAgg", do you mean "qtagg"?, there is no such backend called "PyAgg".

If you control the installed packages, I suppose you could include qt bindings... though that does seem a little silly to bundle Qt in a non-Qt app... (plus licensing when shipping Qt gets more complicated, I'll leave it at that, I am not a lawyer).

I did also try tkagg, as tk is (often, though technically optional) included in base Python installs, but that doesn't even show the plot in the first place and just exits as:

[NSApplication macOSVersion]: unrecognized selector sent to instance 0x12f7291c0

And webagg does kind of work as well, at least it opens in the browser, but there is no way of closing the webserver and the application window is just frozen with the spinning beachball of death. So not ideal, but may have places to push on to make the experience better (e.g. running in a background thread may unlock the UI, then maybe could be some way to tell that the user provided script is still running and signal it to stop)

@prniii
Copy link
Author

prniii commented Oct 21, 2023 via email

@ksunden
Copy link
Member

ksunden commented Oct 30, 2024

Took a brief look at this following discussions in #28981.

The simple act of using the suggested line in close changed behavior, but did not fully resolve it. The app did not fail immediately on closing the window, however attempting to click the button again to display a new plot failed with AccessError in the line which runs python code.

We do not want to hold up the 3.10 release further at this time, so just documenting that I tried it to help with future looking at this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants