Oh goody, we had two tests with the same name, so only the second ran (luckily it now succeeds here, and it's only a smoke test).

Anyone know a good way to check for this within our test suite--besides checking files with seemingly low test coverage %?

More fun:

• Completely duplicated test methods in Test_detrend in test_mlab.py
• In test_pickle.py, recursive_pickle (and hence depth_getter) are completely unused in living code--they're commented out debugging helpers. Suggestions on where else we could put them? Or can we remove?

For top-level functions, this is relatively easy:

$ grep -IRo '^def[[:space:]]\+[A-Za-z0-9_]\+' | sort | uniq -d
test_axes.py:def test_pcolorargs
test_contour.py:def test_contour_manual_labels

For class methods, maybe a short Python script:

import sys
from collections import Counter

for name in sys.argv[1:]:
    methods = Counter()
    class_name = None
    with open(name, 'r') as f:
        for line in f:
           if line.startswith('class'):
               class_name = line.split()[1].split('(')[0].strip()
           elif line.startswith('def'):
               class_name = None
           elif class_name and line.startswith('    def'):
               method_name = line.split()[1].split('(')[0].strip()
               methods[class_name + '.' + method_name] += 1
    duplicates = [name for name, count in methods.most_common() if count != 1]
    if duplicates:
        print(name, duplicates)

test_mlab.py ['Test_detrend.test_detrend_mean_2D_axism1', 'Test_detrend.test_detrend_mean_2D_none', 'Test_detrend.test_detrend_mean_2D_axis0', 'Test_detrend.test_detrend_str_linear_2d_slope_off_axis0', 'Test_detrend.test_detrend_mean_2D_none_T', 'Test_detrend.test_detrend_mean_2D_axis1', 'Test_detrend.test_detrend_detrend_linear_1d_slope_off_axis1']
test_skew.py ['SkewXTick.tick2On', 'SkewXTick.gridOn', 'SkewXTick.tick1On', 'SkewXTick.label2On', 'SkewXTick.label1On']

For top-level classes, too:

$ grep -IRo '^class[[:space:]]\+[A-Za-z0-9_]\+' | sort | uniq -d
test_ticker.py:class TestLogFormatter

I am 👍 on removing the pickle debugging helpers. I have 0 pity on people depending on helper functions from in our tests (not in testing, in the tests). We have already broken those folks by default by not including the tests anyway.

It's been a good exercise. All of the test files with significant uncovered lines were caused by either dead code or tests that weren't running.

I noticed a significant number of uncovered lines left are caused by pass. Thoughts on adding pass to our exclude list in .coveragerc? Or is that cheating?

Another question: Currently, our matplotlib.testing stands at ~57% coverage because all the nose stuff is disabled. Do we want to exclude the nose testing code from the report?

I think both solutions are fine, with a slight preference in keeping it.
The day we will get rid of this code, our coverage will just jump up.

Anyone have opinions on whether we should be ignoring pass lines for test coverage? We currently exclude:

    raise NotImplemented
    def __str__
    def __repr__
    if __name__ == .__main__.:

I don't think we should skip pass lines. Either we have non-functional code (as in the body is just pass and it really should be NotImplemented or they pass lines are in fall-troughs of input sanitation/validation which if we miss them, there is a case of inputs we are missing in the tests.

I think a lot of these are stubs in interfaces in the tests. I'm leaning towards turning them into some variety of NotImplemented such that they break should they be used.