FigureCanvasTkAgg.draw() silently crashes in threaded application #13293

mishioo · 2019-01-26T21:07:26Z

Bug report

Bug summary

Matplotlib crashes python without rising any exceptions (python simply closes), when draw method is called on FigureCanvasTkAgg object in threaded application from thread other than main thread.

Code for reproduction

from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg
from matplotlib.figure import Figure
import tkinter as tk
from threading import Thread

def draw(canvas):
    t = Thread(target=canvas.draw)
    t.start()

root = tk.Tk()
fig = Figure()
canvas = FigureCanvasTkAgg(fig, master=root)
canvas.get_tk_widget().pack()
tk.Button(root, text='Do it!', command=lambda: draw(canvas)).pack()
root.mainloop()

Actual outcome

When button from example shown above is clicked, application window freezes for a moment and than closes along with python, no exception or error is displayed.

Expected outcome

Preferably draw method should run normally, or at least an exception should be rised.
In Matplotlib 2.2.2 same code snipped runs flawlessly.

Matplotlib version

Operating system: Windows 10 Home x64
Matplotlib version: 3.0.2
Matplotlib backend (print(matplotlib.get_backend())): TkAgg
Python version: 3.6.7
Other libraries: tkinter, threading

Matplotlib installed with 'pip install matplotlib'.

The text was updated successfully, but these errors were encountered:

QuLogic · 2019-01-27T01:13:12Z

Matplotlib makes no guarantees of thread safety. You'll need to use other methods to ensure that only one thread accesses it (conditions, mutexes, GUI-specific signals (if available), etc.)

tacaswell · 2019-01-28T01:28:57Z

19:59 $ python -X faulthandler  /tmp/tk_thread.py 
Fatal Python error: Segmentation fault

Current thread 0x00007f7029ab2700 (most recent call first):
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/site-packages/matplotlib/backends/_backend_tk.py", line 91 in blit
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/site-packages/matplotlib/backends/backend_tkagg.py", line 10 in draw
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/threading.py", line 864 in run
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/threading.py", line 884 in _bootstrap

Thread 0x00007f7040fff740 (most recent call first):
  File "/home/tcaswell/mc3/envs/dd36/lib/python3.6/tkinter/__init__.py", line 1277 in mainloop
  File "/tmp/tk_thread.py", line 15 in <module>
Segmentation fault (core dumped)

Even though we do not claim that Matplotlib is threadsafe, there is no obvious race here and the sefault is coming from inside

matplotlib/src/_tkagg.cpp

Lines 212 to 250 in 5a53f5e

    
           static PyObject *mpl_tk_blit(PyObject *self, PyObject *args) 
        
           { 
        
               Tcl_Interp *interp; 
        
               char const *photo_name; 
        
               int height, width; 
        
               unsigned char *data_ptr; 
        
               int o0, o1, o2, o3; 
        
               int x1, x2, y1, y2; 
        
               Tk_PhotoHandle photo; 
        
               Tk_PhotoImageBlock block; 
        
               if (!PyArg_ParseTuple(args, "O&s(iiO&)(iiii)(iiii):blit", 
        
                                     convert_voidptr, &interp, &photo_name, 
        
                                     &height, &width, convert_voidptr, &data_ptr, 
        
                                     &o0, &o1, &o2, &o3, 
        
                                     &x1, &x2, &y1, &y2)) { 
        
                   goto exit; 
        
               } 
        
               if (!(photo = TK_FIND_PHOTO(interp, photo_name))) { 
        
                   PyErr_SetString(PyExc_ValueError, "Failed to extract Tk_PhotoHandle"); 
        
                   goto exit; 
        
               } 
        
               block.pixelPtr = data_ptr + 4 * ((height - y2) * width + x1); 
        
               block.width = x2 - x1; 
        
               block.height = y2 - y1; 
        
               block.pitch = 4 * width; 
        
               block.pixelSize = 4; 
        
               block.offset[0] = o0; 
        
               block.offset[1] = o1; 
        
               block.offset[2] = o2; 
        
               block.offset[3] = o3; 
        
               TK_PHOTO_PUT_BLOCK_NO_COMPOSITE( 
        
                   photo, &block, x1, height - y2, x2 - x1, y2 - y1); 
        
           exit: 
        
               if (PyErr_Occurred()) { 
        
                   return NULL; 
        
               } else { 
        
                   Py_RETURN_NONE; 
        
               } 
        
           }

so I am a bit worried that this is exposing a latent bug.

We may also be hitting non-thread safe operations in tk?

QuLogic · 2019-01-28T10:00:08Z

You can get a backtrace via catchsegv python ../mpl-tests/issue13293.py:

.../conda/envs/mpl36/lib/python3.6/lib-dynload/../../libtk8.6.so(Tk_GetImageMasterData+0x11)[0x7f4d018b9821]
.../conda/envs/mpl36/lib/python3.6/lib-dynload/../../libtk8.6.so(Tk_FindPhoto+0xe)[0x7f4d018c4bee]
.../matplotlib/lib/matplotlib/backends/_tkagg.cpython-36m-x86_64-linux-gnu.so(+0x279e)[0x7f4d11ef379e]

Looking upstream at this comment, there does appear to be some queuing involved and using the interpreter across threads is risky. Our _tkagg C code probably does not do that queuing.

Also according to a later comment, Tk may or may not be compiled with thread support.

xzores · 2020-09-24T19:58:01Z

I also have this problem, calling everything from the same thread.
Newest matlablib and python, really tiered of it

anntzer · 2020-09-24T20:31:21Z

AFAICT this is simply not supported by Tcl. Per http://www.tcl.tk/doc/howto/thread_model.html:

each interpreter is tightly bound to its OS thread and errors will occur if you let more than one thread call into the same interpreter (e.g., with Tcl_Eval).

We could start peppering the codebase with if threading.current_thread() == threading.main_thread() (in this specific case I guess in _backend_tk.blit, but I'm not sure it's so cheap? And we are explicitly not threadsafe anyways (there are other places which I know have race conditions, e.g. likely most things involving _setattr_cm).

So I'll close as wontfix, but as usual feel free to disagree and reopen...

tacaswell · 2020-09-24T20:46:25Z

@xzores Is that thread the main thread? In general GUI toolkits do not like having any part of them be run in a background thread (unless they manage it).

xzores · 2020-09-24T20:47:17Z

No it is in an other thread, but it is the same thread

xzores · 2020-09-24T20:48:12Z

I can't do "if threading.current_thread() == threading.main_thread()", because the main thread is used for tk.mainloop().
So I will never call matlablib from the main thread.
I just want to update a plot. The only way I found is to remove the plot entirely and then remake it, but that is not optimal, for many reasons

tacaswell · 2020-09-24T20:50:59Z

and I agree with the "can't fix" label, getting into thread safetly is starting to get into application logic and I suspect that any attempts on our part would invariably conflict with user's attempts to do the same in their application.

tacaswell · 2020-09-24T20:51:35Z

Use Tk's signaling system to ask the main thread to schedule a draw.

[edit to fix spelling]

xzores · 2020-09-24T20:52:44Z

Use Tk's signally system to ask the main thread to schedule a draw.

I get some data from the network, that I need to append to the existing data and update the plot

xzores · 2020-09-24T20:54:31Z

Calling figure.canvas.draw() crashes my app, no exceptions thrown (that must be a bug)

richardsheridan · 2020-09-24T20:54:53Z

I've actually been working on this for some time. Since everyone seems to be looking at this right now, I've made a PR (#18565) to discuss.

In short, we can use the tkinter cross-thread signaling to make draw and blit thread "safe" but Tk_photoputblock is fundamentally not parallelizable, so this only really prevents crashes.

dstansby added the GUI: tk label Jan 29, 2019

anntzer closed this as completed Sep 24, 2020

anntzer added the status: won't or can't fix label Sep 24, 2020

richardsheridan mentioned this issue Sep 24, 2020

Make Tkagg blit thread safe #18565

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FigureCanvasTkAgg.draw() silently crashes in threaded application #13293

FigureCanvasTkAgg.draw() silently crashes in threaded application #13293

mishioo commented Jan 26, 2019

QuLogic commented Jan 27, 2019

tacaswell commented Jan 28, 2019

QuLogic commented Jan 28, 2019

xzores commented Sep 24, 2020

anntzer commented Sep 24, 2020

tacaswell commented Sep 24, 2020

xzores commented Sep 24, 2020 •

edited

Loading

xzores commented Sep 24, 2020 •

edited

Loading

tacaswell commented Sep 24, 2020

tacaswell commented Sep 24, 2020 •

edited

Loading

xzores commented Sep 24, 2020

xzores commented Sep 24, 2020 •

edited

Loading

richardsheridan commented Sep 24, 2020 •

edited

Loading

FigureCanvasTkAgg.draw() silently crashes in threaded application #13293

FigureCanvasTkAgg.draw() silently crashes in threaded application #13293

Comments

mishioo commented Jan 26, 2019

Bug report

QuLogic commented Jan 27, 2019

tacaswell commented Jan 28, 2019

QuLogic commented Jan 28, 2019

xzores commented Sep 24, 2020

anntzer commented Sep 24, 2020

tacaswell commented Sep 24, 2020

xzores commented Sep 24, 2020 • edited Loading

xzores commented Sep 24, 2020 • edited Loading

tacaswell commented Sep 24, 2020

tacaswell commented Sep 24, 2020 • edited Loading

xzores commented Sep 24, 2020

xzores commented Sep 24, 2020 • edited Loading

richardsheridan commented Sep 24, 2020 • edited Loading

xzores commented Sep 24, 2020 •

edited

Loading

xzores commented Sep 24, 2020 •

edited

Loading

tacaswell commented Sep 24, 2020 •

edited

Loading

xzores commented Sep 24, 2020 •

edited

Loading

richardsheridan commented Sep 24, 2020 •

edited

Loading