Thanks for this. This is really helpful.

@matthew-brett wrote:

Would y'all consider carrying something like this approach in the matplotlib source, but not enabled by default, to help building binary wheels?

I'd prefer to give this loads of testing and just have this as the "one way to do it" if we can. We could eliminate tons of hacks in the build script and tons of pain and support using this approach, at the expense of the very minor dynamic linking at import time.

My C fu is weak; is there a way of collecting the typedefs I need from the TCL / Tk headers rather than copying them into the _tkagg.cpp source file (typdefs starting around line 52)?

I don't think so.

My fu for Python C extension modules is also weak; did I configure exceptions and handle references correctly?

I'll read through and try to verify.

My C fu is weak; is there a way of collecting the typedefs I need from the TCL / Tk headers rather than copying them into the _tkagg.cpp source file (typdefs starting around line 52)?

What's wrong with #includeing the header?

What's wrong with #includeing the header?

Because the point of this change is to replace actual direct function calls with function pointers. Numpy, for example, has a special generated header for this exact purpose (__multiarray_api.h), but most C libraries don't.

I was hoping since it's C++ you could use some auto magic, but that might be C++11 only.

auto is static (i.e. compile-time) typing only, so there's no way that could work for runtime type inference. That's the realm of C++ virtual functions/polymorphism.

We don't need runtime typing; we just need the type of the function at compile time so that we can set up a pointer to it.

The return type of dlsym() (and hence _dfunc) is void *; there is absolutely no way for the compiler to help you out here.

I didn't really mean auto, just some "C++ magic" like it. And that something would be typeof or decltype, but both of these come with their own issues (one's a GNU-ism, the other's C++11 only).

@mdboom - addressed your comments - I think. I've tested disabling the linker flags for TCL / Tk libraries in setupext.py - works on OSX Python 2.7, 3.5, Debian Jessie 2.7 and 3.5.

Cool. If this works, we might as well take most of the crazy header-file-hunting out of setupext.py as well.

As for the Windows-specific issues, @cgohlke, any thoughts?

I've done some speculative changes to support Windows, trying to work out how to test them now.

OK - Windows changes are compiling and passing tests on Python 2.7 at least.

I propose to remove the not-dynamic branch, and disable linking the TCL / Tk libs in setupext.py to see if this can work without build-time linking.

Build-time linking disabled, Appveyor tests passing, failure on travis I believe is unrelated, so from my point of view, I think this is ready.

Maybe I am missing something, but how can this work on Windows where the Tcl/Tk symbols are not exported from _tkinter.pyd? Is appveyor testing TkAgg?

>>> from ctypes import *
>>> from ctypes.wintypes import *
>>> windll.kernel32.LoadLibraryW.restype = HMODULE
>>> windll.kernel32.LoadLibraryW.argtypes = [LPCWSTR]
>>> windll.kernel32.GetProcAddress.argtypes = [HMODULE, LPCSTR]
>>> windll.kernel32.GetProcAddress.restype = c_void_p
>>> from tkinter import _tkinter
>>> hdll = windll.kernel32.LoadLibraryW(_tkinter.__file__)
>>> print(windll.kernel32.GetProcAddress(hdll, b"PyInit__tkinter"))
140732219092336
>>> print(windll.kernel32.GetProcAddress(hdll, b"Tcl_CreateCommand"))
None

Quick test with a conda python 3.5 environment on OS X 10.9.5: it works! I was not able to get a working Tk backend without this.
Maybe unrelated to this, but I get lots of warnings while executing:

2016-05-18 10:51:04.230 python[18741:507] setCanCycle: is deprecated.  Please use setCollectionBehavior instead

Travis passed. Appveyor stalled in the build stage on the Py3.5 job. I didn't see anything indicating the build failure was related to the PR, so I am closing and reopening to try again.

Well, I don't have any idea what the problem is, but Appveyor failed to build on Python 3.5 just like last time. Very little output is generated. It seems to stall before actually doing any conda installations.

Appveyor now passing on all Pythons for some reason. Travis failure looks unrelated. Appveyor and Travis tests now testing tkagg import and succeeding. Tests of this branch on OSX at MacPython/matplotlib-wheels finding and importing tkagg correctly (unrelated failures but tests passing on Pythons 2.7 3.4 3.5).

Everything passed and Appveyor showed the tkagg test result correctly, but it looks like Travis either didn't execute the tkagg import test, or swallowed its output:

The command "echo Testing import of tkagg backend
MPLBACKEND="tkagg" python -c 'import matplotlib.pyplot as plt; print(plt.get_backend())'
echo Testing using $NPROC processes

OSX builds now green - successful tkagg import checks for Python 2.7 Python 3.4 Python 3.5.

Eric - I think the missing output is just because the command is in a block with other commands. The output is further down after the whole command input block - e.g. https://travis-ci.org/matplotlib/matplotlib/jobs/132588520#L1018

backported to v1.5.x as b80e0f1

When updating matplotlib for canopy, I am seeing the error "Could not find TCL routines" with 1.5.3.

At first, I suspect an issue in how we build python for canopy, but I have been able to reproduce the issue with python 2.7.12 from python.org and just installing wheels from Gholke website. The failure appears for both 1.5.3 and 2.0.0.

I could also reproduce the issue w/ 1.5.3 using conda (2.7.13 this time)

Hm, that should really be an issue on its own, sorry