[Bug]: Segmentation fault on macOS when using matplotlib.pyplot.ion() in IPython #27808

mphanderson · 2024-02-21T18:44:29Z

Bug summary

When using matplotlib.pyplot.ion() in an instance of IPython, after around 15-30 minutes the IPython instance will exit and "Segmentation fault 11" is printed in the terminal. This occurs with or without additional code execution or interaction with the terminal, and seems to simply happen after some amount of time.

Code for reproduction

import matplotlib.pyplot as plt

plt.ion()

Actual outcome

In [1]: import matplotlib.pyplot as plt

In [2]: plt.ion()
Out[2]: <contextlib.ExitStack at 0x102af0b50>

In [3]: Segmentation fault: 11

To be clear, IPython does not immediately segfault, it happens after upward of 15 minutes of being idle (or being used).

Expected outcome

I would expect the terminal to remain open indefinitely.

Additional information

I have reproduced this bug with both:
Python 3.11.7, IPython 8.21.0, matplotlib 3.8.3
and
Python 3.9.18, IPython 8.18.1, matplotlib 3.8.2

Both versions of Python were installed using homebrew.

I do not recall this happening with the previous version of Sonoma I was using (14.3).
I am using an M1 MacBook Pro.
I have tried uninstalling and reinstalling matplotlib, which made no difference.
My IPython instances that have not used plt.ion() do not experience this behaviour and will remain open indefinitely.

Operating system

macOS Sonoma 14.3.1

Matplotlib Version

3.8.3

Matplotlib Backend

MacOSX

Python version

3.11.7

Jupyter version

No response

Installation

pip

The text was updated successfully, but these errors were encountered:

tacaswell · 2024-02-21T19:30:29Z

If you use the qt backend does this happen?

ksunden · 2024-02-21T20:04:52Z

it happens after upward of 15 minutes of being idle

Oof... that makes it quite a bit harder to test/dig into...

My mac is currently on Sonoma 14.3 (with the 14.3.1 update available)... I can try to validate that the macos version is part of the equation, but the length and uncertainty of testing makes that a little hard to be confident (and AFAIK I can only really go forward, so can't exactly iterate back and forth)

Sonoma 14.3 introduced a deadlock condition (or perhaps rather exposed/made likely) which was reported in #27720 and fixed for the 3.8.3 release in #27755. I mention this mostly just to highlight that there is recent precedence that macos version is in fact an important variable to consider. A quick perusal of the release notes shows a) nothing obviously related to either problem in 14.3 (or 14.4 beta) b) no published notes for 14.3.1. The more info in the updater indicates only a bug fix for text while typing.

As you are using 3.8.3, that particular fix is in and I don't think it should introduce any segfault conditions.

I would be curious on a few fronts:

is it backend dependent (@tacaswell's point above, try using qt, etc. since it's ion related, would need to be an interactive backend)
- if not, this is likely backend specific to the MacOS backend
Does it happen with python (rather than ipython).
- IPython introduces some amount of additional event loop handling, which I am curious to know if it is interfering here (not exactly sure how, but a variable to isolate
Can we get stack traces using import faulthandler; faulthandler.enable() (or python -X faulthandler if IPython is not involved)
- This whould at least help identify what is actually segfaulting.

I've got a process waiting (on 14.3, not 14.3.1) but have not yet replicated the problem (which I'd like to do before ruling things out). If it doesn't happen here after waiting some considerable period, I'll update and see if I can reproduce, but if you are able to test some of these cases, that can help focus my efforts once I'm able to get going..

greglucas · 2024-02-21T20:46:38Z

I just put this on in the background for an hour and could not reproduce the issue from ipython or python prompts.

MacOS: 14.3.1
ipython: 8.17.2
Python: 3.11.7

Edit: I just upgraded to ipython 8.21.0 and that also never segfaulted.

I am on an Intel mac.

mphanderson · 2024-02-21T21:08:55Z

Thanks all for the responses! I've run a few tests over the past hour.

If you use the qt backend does this happen?

After installing PyQt6, and then using import matplotlib; matplotlib.use('QtAgg'), it doesn't seem to crash. So that could be a workaround in the meantime.

Does it happen with python (rather than ipython).

No!

Can we get stack traces using import faulthandler; faulthandler.enable()

Yes! Here is the output:

Python 3.11.7 (main, Dec  4 2023, 18:10:11) [Clang 15.0.0 (clang-1500.1.0.2.5)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.21.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import faulthandler

In [2]: faulthandler.enable()

In [3]: import matplotlib.pyplot as plt

In [4]: plt.ion()
Out[4]: <contextlib.ExitStack at 0x1056bb150>

In [5]: Fatal Python error: Segmentation fault

Thread 0x0000000170547000 (most recent call first):
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/selectors.py", line 566 in select
  File "/opt/homebrew/lib/python3.11/site-packages/prompt_toolkit/eventloop/inputhook.py", line 135 in run_selector
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 982 in run
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1002 in _bootstrap

Thread 0x000000016f53b000 (most recent call first):
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 327 in wait
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/queue.py", line 171 in get
  File "/opt/homebrew/lib/python3.11/site-packages/prompt_toolkit/patch_stdout.py", line 152 in _write_thread
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 982 in run
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1002 in _bootstrap

Thread 0x000000016e15b000 (most recent call first):
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 327 in wait
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 629 in wait
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/core/history.py", line 894 in run
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/core/history.py", line 60 in only_when_enabled
  File "/opt/homebrew/lib/python3.11/site-packages/decorator.py", line 232 in fun
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1002 in _bootstrap

Current thread 0x00000001e1de5c40 (most recent call first):
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/pt_inputhooks/osx.py", line 135 in _stop_on_read
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/pt_inputhooks/osx.py", line 145 in inputhook
  File "/opt/homebrew/lib/python3.11/site-packages/prompt_toolkit/eventloop/inputhook.py", line 149 in select
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 1884 in _run_once
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 607 in run_forever
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 640 in run_until_complete
  File "/opt/homebrew/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 978 in run
  File "/opt/homebrew/lib/python3.11/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1026 in prompt
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/interactiveshell.py", line 839 in prompt_for_code
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/interactiveshell.py", line 896 in interact
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/interactiveshell.py", line 911 in mainloop
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/ipapp.py", line 317 in start
  File "/opt/homebrew/lib/python3.11/site-packages/traitlets/config/application.py", line 1075 in launch_instance
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/__init__.py", line 130 in start_ipython
  File "/opt/homebrew/bin/ipython", line 8 in <module>

Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, matplotlib._c_internal_utils, PIL._imaging, matplotlib._path, kiwisolver._cext, matplotlib._image (total: 18)
Segmentation fault: 11

ksunden · 2024-02-21T22:38:22Z

Thanks for the added info...

Unfortunately I too have as yet been unable to reproduce the issue (I did update macos after ~40 mins of waiting..._

I'm beginning to suspect a few other variables, e.g. compiler version (which can apply to several steps, Python itself, mpl, maybe even numpy)

The env I started with had CPython compiled with clang 14, mpl installed from wheel
After noting your clang version (15) I got a new venv with newly compiled python to test another instance.

That said... None of the threads appear to be actively in MPL code when the segfault happens, which indicates to me that it may in fact be an IPython bug that e.g. requires the OSX input hook to be used, which happens on ion (but not exclusively). My understanding is that IPython will choose which hook to install when a GUI framework is required (and not sooner) and will use their qt hook when qt is used, etc.

The particular line in the "Current Thread" when the segfault occurs is:

https://github.com/ipython/ipython/blob/8b1204b6c4489c786ada294088a3acef7bd53e90/IPython/terminal/pt_inputhooks/osx.py#L135

This is indeed a ctypes function call, which can conceivably segfault.

So my next question is: can the segfault be reproduced without matplotlib?

I think that ipython --gui osx should install the input hook at startup.

If you want to be extra sure that mpl/numpy are not involved, you could try a fresh venv with only ipython (and its hard dependencies) installed.

If it is the case that it happens without mpl involvement, I don't think there is much for us to do here, but hopefully IPython (or perhaps prompt_toolkit) has some insights for you.

mphanderson · 2024-02-22T11:42:09Z

So I did as you suggested @ksunden and tried ipython --gui osx, and yes that also segfaults:

In [1]: import faulthandler

In [2]: faulthandler.enable()

In [3]: Fatal Python error: Segmentation fault

Thread 0x000000016e76b000 (most recent call first):
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/selectors.py", line 566 in select
  File "/opt/homebrew/lib/python3.11/site-packages/prompt_toolkit/eventloop/inputhook.py", line 135 in run_selector
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 982 in run
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1002 in _bootstrap
 
Thread 0x000000016d75f000 (most recent call first):
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 327 in wait
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/queue.py", line 171 in get
  File "/opt/homebrew/lib/python3.11/site-packages/prompt_toolkit/patch_stdout.py", line 152 in _write_thread
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 982 in run
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1002 in _bootstrap

Thread 0x000000016c753000 (most recent call first):
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 327 in wait
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 629 in wait
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/core/history.py", line 894 in run
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/core/history.py", line 60 in only_when_enabled
  File "/opt/homebrew/lib/python3.11/site-packages/decorator.py", line 232 in fun
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 1002 in _bootstrap

Current thread 0x00000001e1de5c40 (most recent call first):
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/pt_inputhooks/osx.py", line 135 in _stop_on_read
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/pt_inputhooks/osx.py", line 145 in inputhook
  File "/opt/homebrew/lib/python3.11/site-packages/prompt_toolkit/eventloop/inputhook.py", line 149 in select
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 1884 in _run_once
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 607 in run_forever
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 640 in run_until_complete
  File "/opt/homebrew/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 978 in run
  File "/opt/homebrew/lib/python3.11/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1026 in prompt
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/interactiveshell.py", line 839 in prompt_for_code
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/interactiveshell.py", line 896 in interact
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/interactiveshell.py", line 911 in mainloop
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/terminal/ipapp.py", line 317 in start
  File "/opt/homebrew/lib/python3.11/site-packages/traitlets/config/application.py", line 1075 in launch_instance
  File "/opt/homebrew/lib/python3.11/site-packages/IPython/__init__.py", line 130 in start_ipython
  File "/opt/homebrew/bin/ipython", line 8 in <module>
Segmentation fault: 11

ksunden · 2024-03-20T18:53:12Z

Closing since the problem is replicated without Matplotlib. If you have not done so already, please report to IPython.

dstansby added the OS: Apple label Feb 21, 2024

ksunden closed this as completed Mar 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Segmentation fault on macOS when using matplotlib.pyplot.ion() in IPython #27808

[Bug]: Segmentation fault on macOS when using matplotlib.pyplot.ion() in IPython #27808

mphanderson commented Feb 21, 2024 •

edited

Loading

tacaswell commented Feb 21, 2024

ksunden commented Feb 21, 2024

greglucas commented Feb 21, 2024 •

edited

Loading

mphanderson commented Feb 21, 2024 •

edited

Loading

ksunden commented Feb 21, 2024

mphanderson commented Feb 22, 2024

ksunden commented Mar 20, 2024

[Bug]: Segmentation fault on macOS when using matplotlib.pyplot.ion() in IPython #27808

[Bug]: Segmentation fault on macOS when using matplotlib.pyplot.ion() in IPython #27808

Comments

mphanderson commented Feb 21, 2024 • edited Loading

Bug summary

Code for reproduction

Actual outcome

Expected outcome

Additional information

Operating system

Matplotlib Version

Matplotlib Backend

Python version

Jupyter version

Installation

tacaswell commented Feb 21, 2024

ksunden commented Feb 21, 2024

greglucas commented Feb 21, 2024 • edited Loading

mphanderson commented Feb 21, 2024 • edited Loading

ksunden commented Feb 21, 2024

mphanderson commented Feb 22, 2024

ksunden commented Mar 20, 2024

mphanderson commented Feb 21, 2024 •

edited

Loading

greglucas commented Feb 21, 2024 •

edited

Loading

mphanderson commented Feb 21, 2024 •

edited

Loading