I've been thinking we should do this for a long time. How does this deal with the fairly heavy requirements of running the matplotlib tests, notably inkscape and ghostscript?

Also -- we should definitely be testing against Python 3.2 (and maybe 3.1) as well.

Here's the latest Travis result for my fork:

-- http://travis-ci.org/msabramo/matplotlib

IIRC, I took Python 3.2 out of the .travis.yml because it was failing and I wanted to get it green first to show that it works.

You can see the 3.1 and 3.2 failures at http://travis-ci.org/#!/msabramo/matplotlib/builds/1577804 -- in case they're easy to fix. Hmmm, does PIL not support Python 3? Does matplotlib require PIL?

It looks like it isn't doing any of the SVG tests due to lack of Inkscape. That's what tripped me up about using Travis previously.

matplotlib master does not require PIL, though it will use it if available. It looks like these failures are just from installing PIL -- let's not do that ;)

And I should say on further reflection -- let's not let the lack of SVG testing hold this up entirely -- it's still quite useful to do something, even if not everything.

Ok I made the Travis build only use PIL for Python 2 (aside: could also look at Pillow and see if that works with Python 3...). I didn't try doing anything about Inkscape.

Python 3 builds are failing with an error. Can you make sense of this? http://travis-ci.org/#!/msabramo/matplotlib/jobs/1593331

And here are a few errors that I get when testing matplotlib with tox and Python 3.2: https://gist.github.com/2911160 -- let me know if you know how to solve those -- especially the ones where the images don't match.

I'm not sure what the Python 3 build failures are about: some sort of interaction with pip that I don't understand -- note the traceback doesn't show matplotlib at any point, so it's just having a problem with the matplotlib source tree itself for some reason.

Python 3 doesn't get a lot of testing, which makes this work you're doing very important.

I'm able to reproduce some of these failures. The bugs have PR's #943, #945, and #946.

I am not able to reproduce the arc_ellipse problem. Are you able to put up result_images/test_axes/arc_ellipse.* somewhere?

Yeah they don't look very much like ellipses :-)

• http://marc-abramowitz.com/images/arc_ellipse.pdf
• http://marc-abramowitz.com/images/arc_ellipse.png
• http://marc-abramowitz.com/images/arc_ellipse.svg

Aha! I think this may be resolved by #943 -- one test was having a side effect on other tests by leaving a figure behind. Which other test it actually affects is dependent on dictionary ordering. You might trying merging #943, #945 and #946 and see if that resolves your issues (or just wait until they're merged into master -- I don't think they'll take too long to be reviewed.)

FYI: All of the related fixes are merged.

After pulling down your latest fixes, I got my setup on Ubuntu 11.10 down to only 2 failures, which I suspect are related to me not having the correct fonts:

https://gist.github.com/2920271

Here are the resulting images:

• http://www.flickr.com/photos/msabramo/7366707542/
• http://www.flickr.com/photos/msabramo/7181480039/

Do you know what packages I need to install on Ubuntu to get the needed fonts? I've already installed a few that sounded promising, but I'm still getting warnings and failures.

On second thought, it looks like matplotlib includes a bunch of ttf fonts and it should be picking them up out of there but it's not for some reason. I hacked in a little bit of extra debugging info into my working copy to print best_font and best_score (I can submit a PR for this if you want) and here's the results:

/home/marca/dev/git-repos/matplotlib/.tox/py26/lib/python2.6/site-packages/matplotlib/font_manager.py:1215: UserWarning: findfont: Font family ['sans-serif'] not found. Falling back to Bitstream Vera Sans; best_font = None; best_score = 1e+64
  '; best_font = %r; best_score = %r' % (best_font, best_score)
/home/marca/dev/git-repos/matplotlib/.tox/py26/lib/python2.6/site-packages/matplotlib/font_manager.py:1227: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=14.0. Returning /home/marca/dev/git-repos/matplotlib/.tox/py27/local/lib/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf/Vera.ttf; best_font = None; best_score = 1e+64
  UserWarning
/home/marca/dev/git-repos/matplotlib/.tox/py26/lib/python2.6/site-packages/matplotlib/font_manager.py:1227: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=bold:stretch=500:size=14.0. Returning /home/marca/dev/git-repos/matplotlib/.tox/py27/local/lib/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf/Vera.ttf; best_font = None; best_score = 1e+64
  UserWarning
/home/marca/dev/git-repos/matplotlib/.tox/py26/lib/python2.6/site-packages/matplotlib/font_manager.py:1215: UserWarning: findfont: Font family ['sans serif'] not found. Falling back to Bitstream Vera Sans; best_font = None; best_score = 1e+64
  '; best_font = %r; best_score = %r' % (best_font, best_score)
/home/marca/dev/git-repos/matplotlib/.tox/py26/lib/python2.6/site-packages/matplotlib/font_manager.py:1227: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=italic:variant=normal:weight=750:stretch=500:size=14.0. Returning /home/marca/dev/git-repos/matplotlib/.tox/py27/local/lib/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf/Vera.ttf; best_font = None; best_score = 1e+64
  UserWarning
/home/marca/dev/git-repos/matplotlib/.tox/py26/lib/python2.6/site-packages/matplotlib/font_manager.py:1227: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=200:stretch=500:size=14.0. Returning /home/marca/dev/git-repos/matplotlib/.tox/py27/local/lib/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf/Vera.ttf; best_font = None; best_score = 1e+64
  UserWarning
/home/marca/dev/git-repos/matplotlib/.tox/py26/lib/python2.6/site-packages/matplotlib/font_manager.py:1227: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=500:stretch=100:size=14.0. Returning /home/marca/dev/git-repos/matplotlib/.tox/py27/local/lib/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf/Vera.ttf; best_font = None; best_score = 1e+64
  UserWarning

This is a shot in the dark, but can you try removing the font caches in ~/.matplotlib/fontList.cache and ~/.matplotlib/fontList.py3k.cache?

Yes, that worked! Thank you!

GLOB test summary 
  py26: commands succeeded
  py27: commands succeeded
  py32: commands succeeded
  congratulations :)

See #948.

Superceded by #948