Forget it. The relative paths only work with the pep8 tests that we don't want

Looks like codecov.io works much better. For those interested it is running on my master branch here https://codecov.io/github/jenshnielsen/matplotlib

I still have to do some fine tuning before this is ready to merge/review

Just curious -- I've only ever used coveralls, but have found it to be unresponsive at times. Any other advantages to codecov.io?

I spent a significant amount of time at the Scipy sprint trying to get the relative/absolute links to the source working correctly with coveralls. The answer is probably burried somewhere in our setup/installation but with codecov it just seems to work.

There are other issues at the moment. I had to manually exclude the source code of numpy etc. which should already be excluded.

@jenshnielsen: This is cool -- I'd like to start improving coverage, and this is a logical first step. What do you think remains to be done here? I have no problem going with codecov over coveralls if that's working better.

AFAIK it's basically working. I decided to postpone it till after 1.5 but I should be ready to get back to it. Let me see if I can rebase it over the weekend.

This is looking quite cool. I assume when all is said and done, there is a way to not print the coverage report to the console on Travis? It makes it hard to scroll up to find any actual failures or errors.

I think this should finally be ready to go. I guess the main question is if we want to do this or wait for the py.test work which will change this and hopefully make it a lot simpler.

I have moved the environment to init.py and made it possible to run coverage from matplotlib.test() via a function argument. I have reverted to coveralls which seems more stabile than codecov.

The only way have been able to make either of codecov and coveralls able to figure out the files is to do a develop install. This also means that the log is a lot shorted because al the files are not copied.
I have done that via pip install -e . which means that no log at all is shown on a successful build.

Furthermore this is running my fork of nose to be able to suppress the output to std out.

According to the nose docs and various issues you only get reliable coverage output when you disable multiprocessing so I have done that on one of the jobs and only run coverage on that.

With respect for waiting for py.test -- I don't see a strong reason to wait given that you've already done the work here. But I'll leave this up for a while in case there are differing opinions. I agree it should hopefully be easier under py.test in the long run.

I am 👍 on merging this now.

Just noticed that the mpl_toolkits aren't "covered".

I should clarify that what I mean is that the mpl_toolkits, while tested (somewhat), aren't included in any metrics or reporting.

cbook.py also has some really funny looking coverage results.

@WeatherGod I don't think mpl_toolkits are included in @tacaswell original work. I think we just need to add it to the list of modules covered.

Yes there is something odd in cbook and other places. Functions that are not recorded as called but have lines within them registered as called. I will try to reproduce locally.

I am going to merge this now and iterate on it as we go.