Skip to content

Commit 0d5a02a

Browse files
dstansbydstansby
authored
Merge pull request #8026 from QuLogic/pytest-tweaks
Pytest documentation + build tweaks
2 parents 0497a1d + 29c1308 commit 0d5a02a

22 files changed

+199
-229
lines changed

‎.travis.yml

-2
Original file line numberDiff line numberDiff line change
@@ -56,8 +56,6 @@ matrix:
5656
env: MOCK=mock NUMPY=numpy==1.7.1
5757
- python: 3.4
5858
env: PYTHON_ARGS=-OO
59-
- python: 3.5
60-
env: PANDAS=pandas DELETE_FONT_CACHE=1
6159
- python: 3.5
6260
env: BUILD_DOCS=true
6361
- python: 3.5

‎INSTALL

+9-8
Original file line numberDiff line numberDiff line change
@@ -106,14 +106,15 @@ or example code.
106106
If you want to try the many demos that come in the matplotlib source
107107
distribution, download the :file:`*.tar.gz` file and look in the
108108
:file:`examples` subdirectory.
109-
To run the test suite, copy the :file:`lib\\matplotlib\\tests` and
110-
:file:`lib\\mpl_toolkits\\tests` directories from the source
111-
distribution to :file:`sys.prefix\\Lib\\site-packages\\matplotlib` and
112-
:file:`sys.prefix\\Lib\\site-packages\\mpl_toolkits` respectively, and
113-
install `nose <https://pypi.python.org/pypi/nose>`_, `mock
114-
<https://pypi.python.org/pypi/mock>`_, Pillow, MiKTeX, GhostScript,
115-
ffmpeg, avconv, mencoder, ImageMagick, and `Inkscape
116-
<https://inkscape.org/>`_.
109+
To run the test suite:
110+
111+
* extract the :file:`lib\\matplotlib\\tests` or
112+
:file:`lib\\mpl_toolkits\\tests` directories from the source distribution;
113+
* install test dependencies: `pytest <https://pypi.python.org/pypi/pytest>`_,
114+
`mock <https://pypi.python.org/pypi/mock>`_, Pillow, MiKTeX, GhostScript,
115+
ffmpeg, avconv, mencoder, ImageMagick, and `Inkscape
116+
<https://inkscape.org/>`_;
117+
* run ``py.test path\\to\\tests\\directory``.
117118

118119

119120

‎README.rst

+3-4
Original file line numberDiff line numberDiff line change
@@ -20,17 +20,16 @@ Testing
2020

2121
After installation, you can launch the test suite::
2222

23-
python tests.py
23+
py.test
2424

2525
Or from the Python interpreter::
2626

2727
import matplotlib
2828
matplotlib.test()
2929

3030
Consider reading http://matplotlib.org/devel/coding_guide.html#testing for
31-
more information. Note that the test suite requires nose and on Python 2.7 mock
32-
which are not installed by default. Please install with pip or your package
33-
manager of choice.
31+
more information. Note that the test suite requires pytest and, on Python 2.7,
32+
mock. Please install with pip or your package manager of choice.
3433

3534
Contact
3635
=======

‎ci/conda_recipe/meta.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ requirements:
3030
- freetype 2.6*
3131
- msinttypes # [win]
3232
- cycler >=0.10
33-
- nose
33+
- pytest >=3.0.0
3434
- pyparsing
3535
- pytz
3636
# - py2cairo # [linux and py2k]

‎doc/devel/contributing.rst

+6-4
Original file line numberDiff line numberDiff line change
@@ -120,14 +120,16 @@ environment is set up properly::
120120
python tests.py
121121

122122

123-
.. _nose: https://nose.readthedocs.io/en/latest/
123+
.. _pytest: http://doc.pytest.org/en/latest/
124124
.. _pep8: https://pep8.readthedocs.io/en/latest/
125+
.. _mock: https://docs.python.org/dev/library/unittest.mock.html
126+
.. _Ghostscript: https://www.ghostscript.com/
127+
.. _Inkscape: https://inkscape.org>
125128

126129
.. note::
127130

128-
**Additional dependencies for testing**: nose_ (version 1.0 or later), `mock
129-
<https://docs.python.org/dev/library/unittest.mock.html>`_ (if python < 3.3), `Ghostscript
130-
<https://www.ghostscript.com/>`_, `Inkscape <https://inkscape.org>`_
131+
**Additional dependencies for testing**: pytest_ (version 3.0 or later),
132+
mock_ (if python < 3.3), Ghostscript_, Inkscape_
131133

132134
.. seealso::
133135

‎doc/devel/testing.rst

+72-78
Original file line numberDiff line numberDiff line change
@@ -4,31 +4,36 @@
44
Developer's tips for testing
55
============================
66

7-
Matplotlib has a testing infrastructure based on nose_, making it easy
8-
to write new tests. The tests are in :mod:`matplotlib.tests`, and
9-
customizations to the nose testing infrastructure are in
10-
:mod:`matplotlib.testing`. (There is other old testing cruft around,
11-
please ignore it while we consolidate our testing to these locations.)
12-
13-
.. _nose: https://nose.readthedocs.io/en/latest/
7+
Matplotlib's testing infrastructure depends on pytest_. The tests are in
8+
:file:`lib/matplotlib/tests`, and customizations to the pytest testing
9+
infrastructure are in :mod:`matplotlib.testing`.
10+
11+
.. _pytest: http://doc.pytest.org/en/latest/
12+
.. _mock: https://docs.python.org/dev/library/unittest.mock.html>
13+
.. _Ghostscript: https://www.ghostscript.com/
14+
.. _Inkscape: https://inkscape.org
15+
.. _pytest-cov: https://pytest-cov.readthedocs.io/en/latest/
16+
.. _pytest-pep8: https://pypi.python.org/pypi/pytest-pep8
17+
.. _pytest-xdist: https://pypi.python.org/pypi/pytest-xdist
18+
.. _pytest-timeout: https://pypi.python.org/pypi/pytest-timeout
1419

1520
Requirements
1621
------------
1722

1823
The following software is required to run the tests:
1924

20-
- nose_, version 1.0 or later
21-
- `mock <https://docs.python.org/dev/library/unittest.mock.html>`_, when running python
22-
versions < 3.3
23-
- `Ghostscript <https://www.ghostscript.com/>`_ (to render PDF
24-
files)
25-
- `Inkscape <https://inkscape.org>`_ (to render SVG files)
25+
- pytest_, version 3.0.0 or later
26+
- mock_, when running Python versions < 3.3
27+
- Ghostscript_ (to render PDF files)
28+
- Inkscape_ (to render SVG files)
2629

2730
Optionally you can install:
2831

29-
- `coverage <https://coverage.readthedocs.io/en/latest/>`_ to collect coverage
30-
information
31-
- `pep8 <http://pep8.readthedocs.io/en/latest>`_ to test coding standards
32+
- pytest-cov_ to collect coverage information
33+
- pytest-pep8_ to test coding standards
34+
- pytest-timeout_ to limit runtime in case of stuck tests
35+
- pytest-xdist_ to run tests in parallel
36+
3237

3338
Building matplotlib for image comparison tests
3439
----------------------------------------------
@@ -53,58 +58,64 @@ value.
5358
Running the tests
5459
-----------------
5560

56-
Running the tests is simple. Make sure you have nose installed and run::
61+
Running the tests is simple. Make sure you have pytest installed and run::
62+
63+
py.test
64+
65+
or::
5766

5867
python tests.py
5968

6069
in the root directory of the distribution. The script takes a set of
6170
commands, such as:
6271

6372
======================== ===========
64-
``--pep8`` pep8 checks
65-
``--no-pep8`` Do not perform pep8 checks
66-
``--no-network`` Disable tests that require network access
73+
``--pep8`` Perform pep8 checks (requires pytest-pep8_)
74+
``-m "not network"`` Disable tests that require network access
6775
======================== ===========
6876

69-
Additional arguments are passed on to nosetests. See the nose
70-
documentation for supported arguments. Some of the more important ones are given
71-
here:
77+
Additional arguments are passed on to pytest. See the pytest documentation for
78+
`supported arguments`_. Some of the more important ones are given here:
7279

7380
============================= ===========
7481
``--verbose`` Be more verbose
75-
``--processes=NUM`` Run tests in parallel over NUM processes
76-
``--process-timeout=SECONDS`` Set timeout for results from test runner process
77-
``--nocapture`` Do not capture stdout
82+
``--n NUM`` Run tests in parallel over NUM
83+
processes (requires pytest-xdist_)
84+
``--timeout=SECONDS`` Set timeout for results from each test
85+
process (requires pytest-timeout_)
86+
``--capture=no`` or ``-s`` Do not capture stdout
7887
============================= ===========
7988

80-
To run a single test from the command line, you can provide a
81-
dot-separated path to the module followed by the function separated by
82-
a colon, e.g., (this is assuming the test is installed)::
89+
To run a single test from the command line, you can provide a file path,
90+
optionally followed by the function separated by two colons, e.g., (tests do
91+
not need to be installed, but Matplotlib should be)::
8392

84-
python tests.py matplotlib.tests.test_simplification:test_clipping
93+
py.test lib/matplotlib/tests/test_simplification.py::test_clipping
94+
95+
or, if tests are installed, a dot-separated path to the module, optionally
96+
followed by the function separated by two colons, such as::
97+
98+
py.test --pyargs matplotlib.tests.test_simplification::test_clipping
8599

86100
If you want to run the full test suite, but want to save wall time try
87101
running the tests in parallel::
88102

89-
python tests.py --nocapture --verbose --processes=5 --process-timeout=300
103+
py.test --verbose -n 5
90104

105+
Depending on your version of Python and pytest-xdist, you may need to set
106+
``PYTHONHASHSEED`` to a fixed value when running in parallel::
91107

92-
An alternative implementation that does not look at command line
93-
arguments works from within Python is to run the tests from the
94-
matplotlib library function :func:`matplotlib.test`::
108+
PYTHONHASHSEED=0 py.test --verbose -n 5
109+
110+
An alternative implementation that does not look at command line arguments
111+
and works from within Python is to run the tests from the Matplotlib library
112+
function :func:`matplotlib.test`::
95113

96114
import matplotlib
97115
matplotlib.test()
98116

99-
.. hint::
100-
101-
To run the tests you need to install nose and mock if using python 2.7::
102117

103-
pip install nose
104-
pip install mock
105-
106-
107-
.. _`nosetest arguments`: http://nose.readthedocs.io/en/latest/usage.html
118+
.. _supported arguments: http://doc.pytest.org/en/latest/usage.html
108119

109120

110121
Writing a simple test
@@ -113,30 +124,20 @@ Writing a simple test
113124
Many elements of Matplotlib can be tested using standard tests. For
114125
example, here is a test from :mod:`matplotlib.tests.test_basic`::
115126

116-
from nose.tools import assert_equal
117-
118127
def test_simple():
119128
"""
120129
very simple example test
121130
"""
122-
assert_equal(1+1,2)
123-
124-
Nose determines which functions are tests by searching for functions
125-
beginning with "test" in their name.
131+
assert 1 + 1 == 2
126132

127-
If the test has side effects that need to be cleaned up, such as
128-
creating figures using the pyplot interface, use the ``@cleanup``
129-
decorator::
133+
Pytest determines which functions are tests by searching for files whose names
134+
begin with ``"test_"`` and then within those files for functions beginning with
135+
``"test"`` or classes beginning with ``"Test"``.
130136

131-
from matplotlib.testing.decorators import cleanup
132-
133-
@cleanup
134-
def test_create_figure():
135-
"""
136-
very simple example test that creates a figure using pyplot.
137-
"""
138-
fig = figure()
139-
...
137+
Some tests have internal side effects that need to be cleaned up after their
138+
execution (such as created figures or modified rc params). The pytest fixture
139+
:func:`~matplotlib.testing.conftest.mpl_test_settings` will automatically clean
140+
these up; there is no need to do anything further.
140141

141142

142143
Writing an image comparison test
@@ -203,24 +204,22 @@ decorator:
203204
Known failing tests
204205
-------------------
205206

206-
If you're writing a test, you may mark it as a known failing test with
207-
the :func:`~matplotlib.testing.decorators.knownfailureif`
208-
decorator. This allows the test to be added to the test suite and run
209-
on the buildbots without causing undue alarm. For example, although
210-
the following test will fail, it is an expected failure::
207+
If you're writing a test, you may mark it as a known failing test with the
208+
:func:`pytest.mark.xfail` decorator. This allows the test to be added to the
209+
test suite and run on the buildbots without causing undue alarm. For example,
210+
although the following test will fail, it is an expected failure::
211211

212-
from nose.tools import assert_equal
213-
from matplotlib.testing.decorators import knownfailureif
212+
import pytest
214213

215-
@knownfailureif(True)
214+
@pytest.mark.xfail
216215
def test_simple_fail():
217216
'''very simple example test that should fail'''
218-
assert_equal(1+1,3)
217+
assert 1 + 1 == 3
219218

220-
Note that the first argument to the
221-
:func:`~matplotlib.testing.decorators.knownfailureif` decorator is a
222-
fail condition, which can be a value such as True, False, or
223-
'indeterminate', or may be a dynamically evaluated expression.
219+
Note that the first argument to the :func:`~pytest.mark.xfail` decorator is a
220+
fail condition, which can be a value such as True, False, or may be a
221+
dynamically evaluated expression. If a condition is supplied, then a reason
222+
must also be supplied with the ``reason='message'`` keyword argument.
224223

225224
Creating a new module in matplotlib.tests
226225
-----------------------------------------
@@ -229,11 +228,6 @@ We try to keep the tests categorized by the primary module they are
229228
testing. For example, the tests related to the ``mathtext.py`` module
230229
are in ``test_mathtext.py``.
231230

232-
Let's say you've added a new module named ``whizbang.py`` and you want
233-
to add tests for it in ``matplotlib.tests.test_whizbang``. To add
234-
this module to the list of default tests, append its name to
235-
``default_test_modules`` in :file:`lib/matplotlib/__init__.py`.
236-
237231
Using Travis CI
238232
---------------
239233

‎doc/users/event_handling.rst

+4-4
Original file line numberDiff line numberDiff line change
@@ -29,13 +29,13 @@ connect your function to the event manager, which is part of the
2929
example that prints the location of the mouse click and which button
3030
was pressed::
3131

32-
fig = plt.figure()
33-
ax = fig.add_subplot(111)
32+
fig, ax = plt.subplots()
3433
ax.plot(np.random.rand(10))
3534

3635
def onclick(event):
37-
print('button=%d, x=%d, y=%d, xdata=%f, ydata=%f' %
38-
(event.button, event.x, event.y, event.xdata, event.ydata))
36+
print('%s click: button=%d, x=%d, y=%d, xdata=%f, ydata=%f' %
37+
('double' if event.dblclick else 'single', event.button,
38+
event.x, event.y, event.xdata, event.ydata))
3939

4040
cid = fig.canvas.mpl_connect('button_press_event', onclick)
4141

‎examples/event_handling/test_mouseclicks.py

-32
This file was deleted.

0 commit comments

Comments
 (0)