Indeed, the transition to py3 isn't complete for this file. The problem arises because urlretrieve() is now under urllib.request instead of just urllib. I'm not sure how to architect a fix, but the following is a diff of the changes needed for it to run under py3, instead of py2.

--- make.osx    2012-03-15 07:30:10.000000000 -0700
+++ make-py3.osx    2012-03-16 00:20:15.000000000 -0700
@@ -56,9 +56,9 @@ clean:


 fetch:
-   ${PYTHON} -c 'import urllib; urllib.urlretrieve("${ZLIBURL}", "${ZLIBFILE}")' &&\
-   ${PYTHON} -c 'import urllib; urllib.urlretrieve("${PNGURL}", "${PNGFILE}")' &&\
-   ${PYTHON} -c 'import urllib; urllib.urlretrieve("${FREETYPEURL}", "${FREETYPEFILE}")'
+   ${PYTHON} -c 'import urllib.request; urllib.request.urlretrieve("${ZLIBURL}", "${ZLIBFILE}")' &&\
+   ${PYTHON} -c 'import urllib.request; urllib.request.urlretrieve("${PNGURL}", "${PNGFILE}")' &&\
+   ${PYTHON} -c 'import urllib.request; urllib.request.urlretrieve("${FREETYPEURL}", "${FREETYPEFILE}")'

 zlib: check-prefix
    rm -rf ${ZLIBDIR} &&\

One could condition on the defined python version so as to choose the proper call, but there's probably a better solution out there.

@mdboom: Does this need to be fixed for 1.2? I am able to build mpl using the setup.py file, but I believe the make.osx is used for building the binary distribution?

Yes -- this should be fixed for 1.2. I'm not an OS-X user, so it would be helpful to know what the advantages of the various ways to build on OS-X are and which need to be kept working. All of them should work on Python 2.6, 2.7, 3.1 and 3.2.

On Aug 19, 2012, at 12:36 PM, Eric Firing wrote:

On 2012/08/19 5:56 AM, Phil Elson wrote:

@mdboom https://github.com/mdboom: Does this need to be fixed for 1.2?
I am able to build mpl using the |setup.py| file, but I believe the
|make.osx| is used for building the binary distribution?

I think that was true at one time--but not any time recently. I believe Russell Owen has been building all our more recent OS X installers, and has been doing so without using make.osx. It may be time to simply delete make.osx and references to it from mpl 1.2.

Yes. I build the binary installers and I use bdist_mpkg (which uses distutils) instead of make.osx.

make.osx was useful because it installed necessary dependencies. However, recent versions of Mac OS X already include all those dependencies, so I am not sure I see a use for it (I never used it anyway, preferring to install dependencies manually).

I'm in favor of removing make.osx. Does anyone else need it?

Regards,

-- Russell

I'm not an OS-X user, so I'm commenting from afar -- but I think anything that can present "one obvious way to do it" here would be helpful. It's a pretty common source of user questions on the mailing list.

I'm an OS X user and build everything from source. I install the dependencies from macports. I also know a lot of people use homebrew for dependencies. As for LaTeX, there's MacTeX, or there's the macports latex port.

Do you use setup.py or make.osx?

Sorry, I meant to say I use setup.py. I also use a custom distutils file to install in my home directory. This, however, clashes with the python ports installed by macports, so I just rename it before installing matplotlib:

mv ~/.pydistutils.MYCFG ~/.pydistutils.cfg
cd ~/git/github/matplotlib
python setup.py install
mv ~/.pydistutils,cfg ~/.pydistutils.MYCFG

Thanks @dmcdougall.

With this final piece of information, I propose we remove make.osx. If somebody comes along with a reasonable use-case for using make.osx instead of setup.py we can either resolve it or ultimately, re-instate make.osx from the repository archive.

Any objections to this approach?

That sounds reasonable -- I'm all for making the best way be more obvious -- but we should probably post this to matplotlib-devel just to give it prominence before removing the file. I'll go ahead and do that.

Great. Thanks Michael.

Before removing it. It would be a good idea to update setupext.py to include paths used by macports for dependencies (if macports is installed). I'd be happy to try and look into this if it's desired. Perhaps we should also do this for homebrew, though I have no experience with homebrew or where it installs things.

Homebrew installs to /usr/local by default.

@dmcdougall: that seems like a good idea. It sounds like homebrew is already covered -- maybe @jkseppan could confim -- and then go ahead and add any macports paths that are necessary.

Yes, homebrew is covered. Also, since I have incompatible libraries in /usr/local, I installed homebrew in /opt/homebrew and set

basedirlist = /opt/homebrew /usr /usr/X11

in setup.cfg, and it built just fine.

To add to what @jkseppan said: Vanilla homebrew is what is recommended (keeping it in /usr/local) by the homebrew project, which just works with mpl out of the box.

@mdboom @jkseppan @pelson Thanks for confirming.

Macports installs into /opt/local/ by default, which is recommended by them. Actually, on further inspection, there seems to be more going on. I currently have gtk+ and pygtk insalled via macports, and the headers are in /opt/local/include/gtk-2.0/. When I compile mpl from scratch it says it can't find the headers for gtk. This is even after I add /opt/local to my basedirlist. This might not be simple to solve.

On Sep 11, 2012, at 11:43 , Damon McDougall wrote:

I currently have gtk+ and pygtk insalled via macports, and the headers are in /opt/local/include/gtk-2.0/. When I compile mpl from scratch it says it can't find the headers for gtk. This is even after I add /opt/local to my basedirlist. This might not be simple to solve.

Isn't this what pkg-config is supposed to solve? One source of problems can be the order in which you have the various directories on $PATH: if you accidentally get the system pkg-config, it doesn't know about MacPorts libraries, so it looks like you don't have the gtk+ headers installed, even though you get some other libraries from /opt/local because setupext.py looks there.

Maybe the instructions need to say something about setting up your $PATH properly. Also, mixing libraries from MacPorts and Homebrew is bound to cause problems, so you should choose one and stick with it.

@jkseppan Thanks for pointing that out, yes pkg-config can't find the headers when I compile from source.

My $PATH is correct:

$ which pkg-config
/opt/local/bin/pkg-config

This is the macports installation of pkg-config. I don't have homebrew installed.

I found the problem. One needs to set PKG_CONFIG_PATH for python versions 2.6+.

Should I put in a pull request with the change to automatically set basedirlist when macports is installed?

I don't know if we need to detect MacPorts in order to add to basedirlist -- we could just add it to the end.

However, is there any way to automate the PKG_CONFIG_PATH stuff? That seems like a tricky hurdle. If not, we'll need to at least document that in the installation instructions.

(I commented via email but it might have gotten lost in the Github outage.)

One failure mode in the OS X build seems to be that pkg-config is not found or does not find the correct *.pc files:

http://stackoverflow.com/questions/12363557/matplotlib-install-failure-on-mac-osx-10-8-mountain-lion/12372968#12372968

In this case, setupext.py first says that pkg-config is not found, but then tries to build anyway. It might be easier if the build stopped right away if pkg-config doesn't find required libraries. Or is there an important use case when people don't have pkg-config but add the compiler flags for the libraries in setup.cfg?

@jkseppan My only guess would be if one wanted a custom build of some library and installed into the home directory. Presumably this is the kind of thing pkg-config wouldn't necessarily pick up and, as you point out, further customisation in setup.cfg would be needed.

My guess is that person forgot to set their $PATH correctly.

On Sep 11, 2012, at 17:15 , Michael Droettboom wrote:

I don't know if we need to detect MacPorts in order to add to basedirlist -- we could just add it to the end.

However, is there any way to automate the PKG_CONFIG_PATH stuff? That seems like a tricky hurdle. If not, we'll need to at least document that in the installation instructions.

—
Reply to this email directly or view it on GitHub.

Should we make failure to call pkg-config a hard failure? See e.g.

http://stackoverflow.com/questions/12363557/matplotlib-install-failure-on-mac-osx-10-8-mountain-lion/12372968#12372968

In this case, setup.py detected (correctly, I suppose) that pkg-config could not locate some libraries, but it tried to continue building anyway, with the resulting build failure. If the initial phase had stopped with "pkg-config not found" or some such error message, it might have been clearer what needs to be installed.

Should we make failure to call pkg-config a hard failure?

If it is truly a build requirement, then yes, I think we should.

Is anyone willing to take this on? @dmcdougall , @jkseppan , @efiring or myself seem like the obvious candidates (I'm a little hard pressed hence not jumping in at this point, but if nobody else can, I'm happy to do it).

Years ago when I added the pkg-config support, it was decided that this would not be a hard requirement, because I think it wasn't there in some environments. That's probably less the case now, but I'm wary of removing a fallback that some may still be unknowingly relying on.

I think I'm going to propose this: Let's cut a 1.2.0rc1 candidate now, and get a tarball out there. This should hopefully encourage people to build it in lots of different Mac environments and any sort of setup.py fixes that are needed for that can be added for rc2 if necessary. Those are the kinds of "build polish" things that are generally easy to do in the rc to final release period.

Sound plan.

+1

Agreed.

Is there any action that needs to be taken here? I haven't heard any complaints on the mailing list about build problems regarding Python 3.2.

However, is there any way to automate the PKG_CONFIG_PATH stuff?

I think so:

from distutils.sysconfig import get_config_var

pkg_config_path = get_config_var('LIBDIR') + '/pkgconfig'

Seem reasonable?