-
-
Notifications
You must be signed in to change notification settings - Fork 7.9k
Prepare for cross-framework test suite #6920
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,15 +1,3 @@ | ||
class KnownFailureTest(Exception): | ||
""" | ||
Raise this exception to mark a test as a known failing test. | ||
""" | ||
|
||
|
||
class KnownFailureDidNotFailTest(Exception): | ||
""" | ||
Raise this exception to mark a test should have failed but did not. | ||
""" | ||
|
||
|
||
class ImageComparisonFailure(AssertionError): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would rather not move this for compatibility reasons. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, I am pretty sure other projects actually uses our KnownFailure stuff. On Wed, Aug 17, 2016 at 1:15 PM, Thomas A Caswell notifications@github.com
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I cannot agree with this for two reasons:
I can understand backward compatibility on matplotlib core features, but with removing nose support all this stuff will gone. I have already tried my best to make dual nose-pytest support, but leaving nose-specific stuff on the current places will blow something in any time in the future. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just to note, matplotlib and many other projects depend on some of numpy's I am not exactly sure how to proceed here. On Wed, Aug 17, 2016 at 1:31 PM, Nikita Kniazev notifications@github.com
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. By my logic - anything that is not a part of the public API can be changed or even removed at any time, so everyone who did something like using There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Numpy's tools are useful for testing using numpy arrays, so I don't find referencing them to be a compelling argument. The exceptions removed here are not unique to matplotlib functionality. It was a bad idea to for people to use them from us in the first place, and are trivially recreated for anyone down stream who happen to use them. |
||
""" | ||
Raise this exception to mark a test as a comparison between two images. | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,70 @@ | ||
from __future__ import (absolute_import, division, print_function, | ||
unicode_literals) | ||
|
||
|
||
def get_extra_test_plugins(): | ||
from .plugins.performgc import PerformGC | ||
from .plugins.knownfailure import KnownFailure | ||
from nose.plugins import attrib | ||
|
||
return [PerformGC, KnownFailure, attrib.Plugin] | ||
|
||
|
||
def get_env(): | ||
env = {'NOSE_COVER_PACKAGE': ['matplotlib', 'mpl_toolkits'], | ||
'NOSE_COVER_HTML': 1, | ||
'NOSE_COVER_NO_PRINT': 1} | ||
return env | ||
|
||
|
||
def check_deps(): | ||
try: | ||
import nose | ||
try: | ||
from unittest import mock | ||
except ImportError: | ||
import mock | ||
except ImportError: | ||
print("matplotlib.test requires nose and mock to run.") | ||
raise | ||
|
||
|
||
def test(verbosity=None, coverage=False, switch_backend_warn=True, **kwargs): | ||
from ... import default_test_modules, get_backend, use | ||
|
||
old_backend = get_backend() | ||
try: | ||
use('agg') | ||
import nose | ||
from nose.plugins import multiprocess | ||
|
||
# Nose doesn't automatically instantiate all of the plugins in the | ||
# child processes, so we have to provide the multiprocess plugin with | ||
# a list. | ||
extra_plugins = get_extra_test_plugins() | ||
multiprocess._instantiate_plugins = extra_plugins | ||
|
||
env = get_env() | ||
if coverage: | ||
env['NOSE_WITH_COVERAGE'] = 1 | ||
|
||
if verbosity is not None: | ||
env['NOSE_VERBOSE'] = verbosity | ||
|
||
success = nose.run( | ||
addplugins=[plugin() for plugin in extra_plugins], | ||
env=env, | ||
defaultTest=default_test_modules, | ||
**kwargs | ||
) | ||
finally: | ||
if old_backend.lower() != 'agg': | ||
use(old_backend, warn=switch_backend_warn) | ||
|
||
return success | ||
|
||
|
||
def knownfail(msg): | ||
from .exceptions import KnownFailureTest | ||
# Keep the next ultra-long comment so it shows in console. | ||
raise KnownFailureTest(msg) # An error here when running nose means that you don't have the matplotlib.testing.nose.plugins:KnownFailure plugin in use. # noqa |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mock
is used in other tests; not sure this test should be removed.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Check is not removed, it is just moved to
check_deps
function