Do not inherit tests from unittest.UnitTest class

As someone who's a big fan of these, what's the rational? Unlike nose, it's part of the standard library and it doesn't seem like it's likely to be deprecated.

There are converters for nose and unittest, though they aren't totally perfect.

Yeah, I guess my question is "will pytest not run unittest" tests? Like I understand dumping nose 'cause it's deprecated, but I don't get the issue with unittest unless it explicitly won't work with pytest

They will work and they are working (pytest handles them with some internal loader hacking in _pytest.unitest). But it is useless import of a module that actuality does not used, and you can do same and even more more with pytest

We would work with @astrofrog to get the features we need in pytest-mpl.

@tacaswell are you sure we want to do that. It's the opposite of what @mdboom suggested in #5405

@tacaswell - I'd be happy to chat about this at SciPy!

Regarding the negatives for pytest-mpl:

‒ need to supply baseline_dir in every decorator

Actually this is not quite correct - there is a default value which is a folder called baseline relative to the test. I can also easily add a global setting.

‒ does not support image format conversion
‒ need something to do with freetype_version, remove_text, and style

I don't think there's any reason we can't add all this, and I don't think it should be that difficult. I'd love to work with you all to make pytest-mpl suitable for matplotlib! (these are all features I would find useful for other projects I use pytest-mpl for)

Another point: we need to get rid of the yield-based tests; they are causing tons of warning in the pytest built.

The yield-based tests are fixed, but do we still have a problem of not being able to use xdist?

The yield-based tests are fixed, but do we still have a problem of not being able to use xdist?

It looks like we can. At least I have tried and it is good. PR #7468 is opened.

In my own project using pytest, attempting to use @image_comparison seems to make the test undetected by pytest. This seems to contradict https://github.com/matplotlib/matplotlib/blob/0d5a02a4e118eb098b78ea00039acd06310e54be/doc/devel/testing.rst. Is it known behaviour?

Are you aware of https://github.com/matplotlib/pytest-mpl ?

@jnothman This will be in the 2.1 release so unless you have installed from source you will not have it.

pytest-mpl is probably a better option.

@jnothman Have you seen the remove_text option for image_comparison?

I think we can close this now. Is that the case?

There are still items unchecked that need to be implemented or decided to ignore.

Items I think we should ignore:

• rewrite image_comparison decorator. (this is an enhancement and clearly not a prerequisite)
• rewrite all assertions to pure assert. (i think numpy.testing.assert_* should stay as-is)

I'm -0 on using pytest-mock (like all testing libraries, its docks assume I already know way more than I do)

I've marked

Rewrite all assertations to pure assert

as done. I agree with @phobson that numpy.testing.* functions should be kept. All other asserts are already written.

Also checked

Rename test_only.py

because there is no such file anymore.

I decided to strikeout the "use pytest-mock" line (agreeing with @phobson above); unittest.mock works just fine in the rare cases where we use it and bringing in a new dependency seems overkill (unless someone has a good use case for it in the test suite, in which case please speak up :)).

The "rewrite image_comparison (currently it slows down collection due to baseline image copying)" item seems not true anymore:

1. image copying only occurs in compare() now, not when collecting.
2. Use symlinks instead of copies for test result_images. #10222 replaces copies by symlinks; at some point it gave a decent speedup but I can't repro it anymore at least locally.

Technically there is one setup_method left in test_transforms.py, but in that specific case it doesn't seem worthwhile to remove it.
I'm thus declaring this issue closed :) Thanks to all for worked on it!