Update documentation to use pytest for testing.

QuLogic · QuLogic · commit 815bc29dc139 · 2017-02-05T20:24:43.000-05:00
diff --git a/INSTALL b/INSTALL
@@ -110,7 +110,7 @@ To run the test suite, copy the :file:`lib\\matplotlib\\tests` and
 :file:`lib\\mpl_toolkits\\tests` directories from the source
 distribution to :file:`sys.prefix\\Lib\\site-packages\\matplotlib` and
 :file:`sys.prefix\\Lib\\site-packages\\mpl_toolkits` respectively, and
-install `nose <https://pypi.python.org/pypi/nose>`_, `mock
+install `pytest <https://pypi.python.org/pypi/pytest>`_, `mock
 <https://pypi.python.org/pypi/mock>`_, Pillow, MiKTeX, GhostScript,
 ffmpeg, avconv, mencoder, ImageMagick, and `Inkscape
 <https://inkscape.org/>`_.
diff --git a/README.rst b/README.rst
@@ -28,9 +28,9 @@ Or from the Python interpreter::
   matplotlib.test()
 
 Consider reading http://matplotlib.org/devel/coding_guide.html#testing for
-more information. Note that the test suite requires nose and on Python 2.7 mock
-which are not installed by default. Please install with pip or your package
-manager of choice.
+more information. Note that the test suite requires pytest and on Python 2.7
+mock which are not installed by default. Please install with pip or your
+package manager of choice.
 
 Contact
 =======
diff --git a/doc/devel/contributing.rst b/doc/devel/contributing.rst
@@ -120,14 +120,16 @@ environment is set up properly::
   python tests.py
 
 
-.. _nose: https://nose.readthedocs.io/en/latest/
+.. _pytest: http://doc.pytest.org/en/latest/
 .. _pep8: https://pep8.readthedocs.io/en/latest/
+.. _mock: https://docs.python.org/dev/library/unittest.mock.html
+.. _Ghostscript: https://www.ghostscript.com/
+.. _Inkscape: https://inkscape.org>
 
 .. note::
 
-  **Additional dependencies for testing**: nose_ (version 1.0 or later), `mock
-  <https://docs.python.org/dev/library/unittest.mock.html>`_ (if python < 3.3), `Ghostscript
-  <https://www.ghostscript.com/>`_, `Inkscape <https://inkscape.org>`_
+  **Additional dependencies for testing**: pytest_ (version 3.0 or later),
+  mock_ (if python < 3.3), Ghostscript_, Inkscape_
 
 .. seealso::
 
diff --git a/doc/devel/testing.rst b/doc/devel/testing.rst
@@ -4,31 +4,34 @@
 Developer's tips for testing
 ============================
 
-Matplotlib has a testing infrastructure based on nose_, making it easy
-to write new tests. The tests are in :mod:`matplotlib.tests`, and
-customizations to the nose testing infrastructure are in
-:mod:`matplotlib.testing`. (There is other old testing cruft around,
-please ignore it while we consolidate our testing to these locations.)
-
-.. _nose: https://nose.readthedocs.io/en/latest/
+Matplotlib has a testing infrastructure based on pytest_, making it easy to
+write new tests. The tests are in :mod:`matplotlib.tests`, and customizations
+to the pytest testing infrastructure are in :mod:`matplotlib.tests.conftest`
+and :mod:`matplotlib.testing`. (There is other old testing cruft around, please
+ignore it while we consolidate our testing to these locations.)
+
+.. _pytest: http://doc.pytest.org/en/latest/
+.. _mock: https://docs.python.org/dev/library/unittest.mock.html>
+.. _Ghostscript: https://www.ghostscript.com/
+.. _Inkscape: https://inkscape.org
+.. _pytest-cov: https://pytest-cov.readthedocs.io/en/latest/
+.. _pytest-pep8: https://pypi.python.org/pypi/pytest-pep8
 
 Requirements
 ------------
 
 The following software is required to run the tests:
 
-  - nose_, version 1.0 or later
-  - `mock <https://docs.python.org/dev/library/unittest.mock.html>`_, when running python
-    versions < 3.3
-  - `Ghostscript <https://www.ghostscript.com/>`_ (to render PDF
-    files)
-  - `Inkscape <https://inkscape.org>`_ (to render SVG files)
+  - pytest_, version 3.0.0 or later
+  - mock_, when running Python versions < 3.3
+  - Ghostscript_ (to render PDF files)
+  - Inkscape_ (to render SVG files)
 
 Optionally you can install:
 
-  - `coverage <https://coverage.readthedocs.io/en/latest/>`_ to collect coverage
-    information
-  - `pep8 <http://pep8.readthedocs.io/en/latest>`_ to test coding standards
+  - pytest-cov_ to collect coverage information
+  - pytest-pep8_ to test coding standards
+
 
 Building matplotlib for image comparison tests
 ----------------------------------------------
@@ -53,58 +56,72 @@ value.
 Running the tests
 -----------------
 
-Running the tests is simple. Make sure you have nose installed and run::
+Running the tests is simple. Make sure you have pytest installed and run::
+
+   py.test
+
+or::
 
    python tests.py
 
 in the root directory of the distribution. The script takes a set of
 commands, such as:
 
 ========================  ===========
-``--pep8``                pep8 checks
-``--no-pep8``             Do not perform pep8 checks
+``--pep8``                Perform pep8 checks (requires pytest-pep8_)
 ``--no-network``          Disable tests that require network access
 ========================  ===========
 
-Additional arguments are passed on to nosetests. See the nose
-documentation for supported arguments. Some of the more important ones are given
-here:
+Additional arguments are passed on to pytest. See the pytest documentation for
+supported arguments. Some of the more important ones are given here:
 
 =============================  ===========
 ``--verbose``                  Be more verbose
-``--processes=NUM``            Run tests in parallel over NUM processes
-``--process-timeout=SECONDS``  Set timeout for results from test runner process
-``--nocapture``                Do not capture stdout
+``--n NUM``                    Run tests in parallel over NUM
+                               processes (requires pytest-xdist_)
+``--timeout=SECONDS``          Set timeout for results from each test
+                               process (requires pytest-timeout_)
+``--capture=no`` or ``-s``     Do not capture stdout
 =============================  ===========
 
-To run a single test from the command line, you can provide a
-dot-separated path to the module followed by the function separated by
-a colon, e.g., (this is assuming the test is installed)::
+To run a single test from the command line, you can provide a dot-separated
+path to the module, optionally followed by the function separated by two
+colons, e.g., (this is assuming the test is installed)::
 
-  python tests.py matplotlib.tests.test_simplification:test_clipping
+  python tests.py matplotlib.tests.test_simplification::test_clipping
+
+or by passing a file path, optionally followed by the function separated by two
+colons, e.g., (tests do not need to be installed, but Matplotlib should be)::
+
+  python tests.py lib/matplotlib/tests/test_simplification.py::test_clipping
 
 If you want to run the full test suite, but want to save wall time try
 running the tests in parallel::
 
-  python tests.py --nocapture --verbose --processes=5 --process-timeout=300
+  python tests.py --capture=no --verbose -n 5
+
+Depending on your version of Python and pytest-xdist, you may need to set
+``PYTHONHASHSEED`` to a fixed value when running in parallel::
 
+  PYTHONHASHSEED=0 python tests.py --capture=no --verbose -n 5
 
-An alternative implementation that does not look at command line
-arguments works from within Python is to run the tests from the
-matplotlib library function :func:`matplotlib.test`::
+An alternative implementation that does not look at command line arguments
+and works from within Python is to run the tests from the Matplotlib library
+function :func:`matplotlib.test`::
 
   import matplotlib
   matplotlib.test()
 
 .. hint::
 
-   To run the tests you need to install nose and mock if using python 2.7::
+   To run the tests you need to install pytest and mock if using python 2.7::
 
-      pip install nose
+      pip install pytest
       pip install mock
 
 
-.. _`nosetest arguments`: http://nose.readthedocs.io/en/latest/usage.html
+.. _pytest-xdist: https://pypi.python.org/pypi/pytest-xdist
+.. _pytest-timeout: https://pypi.python.org/pypi/pytest-timeout
 
 
 Writing a simple test
@@ -113,30 +130,20 @@ Writing a simple test
 Many elements of Matplotlib can be tested using standard tests. For
 example, here is a test from :mod:`matplotlib.tests.test_basic`::
 
-  from nose.tools import assert_equal
-
   def test_simple():
       """
       very simple example test
       """
-      assert_equal(1+1,2)
-
-Nose determines which functions are tests by searching for functions
-beginning with "test" in their name.
-
-If the test has side effects that need to be cleaned up, such as
-creating figures using the pyplot interface, use the ``@cleanup``
-decorator::
+      assert 1 + 1 == 2
 
-  from matplotlib.testing.decorators import cleanup
+Pytest determines which functions are tests by searching for files whose names
+begin with ``"test_"`` and then within those files for functions beginning with
+``"test"`` or classes beginning with ``"Test"``.
 
-  @cleanup
-  def test_create_figure():
-      """
-      very simple example test that creates a figure using pyplot.
-      """
-      fig = figure()
-      ...
+Tests that have side effects that need to be cleaned up, such as created
+figures using the pyplot interface or modified rc params, will be automatically
+reset by the pytest fixture
+:func:`~matplotlib.tests.conftest.mpl_test_settings`.
 
 
 Writing an image comparison test
@@ -203,24 +210,22 @@ decorator:
 Known failing tests
 -------------------
 
-If you're writing a test, you may mark it as a known failing test with
-the :func:`~matplotlib.testing.decorators.knownfailureif`
-decorator. This allows the test to be added to the test suite and run
-on the buildbots without causing undue alarm. For example, although
-the following test will fail, it is an expected failure::
+If you're writing a test, you may mark it as a known failing test with the
+:func:`pytest.mark.xfail` decorator. This allows the test to be added to the
+test suite and run on the buildbots without causing undue alarm. For example,
+although the following test will fail, it is an expected failure::
 
-  from nose.tools import assert_equal
-  from matplotlib.testing.decorators import knownfailureif
+  import pytest
 
-  @knownfailureif(True)
+  @pytest.mark.xfail
   def test_simple_fail():
       '''very simple example test that should fail'''
-      assert_equal(1+1,3)
+      assert 1 + 1 == 3
 
-Note that the first argument to the
-:func:`~matplotlib.testing.decorators.knownfailureif` decorator is a
-fail condition, which can be a value such as True, False, or
-'indeterminate', or may be a dynamically evaluated expression.
+Note that the first argument to the :func:`~pytest.mark.xfail` decorator is a
+fail condition, which can be a value such as True, False, or may be a
+dynamically evaluated expression. If a condition is supplied, then a reason
+must also be supplied with the ``reason='message'`` keyword argument.
 
 Creating a new module in matplotlib.tests
 -----------------------------------------
@@ -229,11 +234,6 @@ We try to keep the tests categorized by the primary module they are
 testing.  For example, the tests related to the ``mathtext.py`` module
 are in ``test_mathtext.py``.
 
-Let's say you've added a new module named ``whizbang.py`` and you want
-to add tests for it in ``matplotlib.tests.test_whizbang``.  To add
-this module to the list of default tests, append its name to
-``default_test_modules`` in :file:`lib/matplotlib/__init__.py`.
-
 Using Travis CI
 ---------------
 
diff --git a/examples/README.txt b/examples/README.txt
@@ -35,7 +35,7 @@ directories found here:
 
   * tests - tests used by matplotlib developers to check functionality
     (These tests are still sometimes useful, but mostly developers should
-    use the nosetests which perform automatic image comparison.)
+    use the pytest tests which perform automatic image comparison.)
 
   * units - working with unit data an custom types in matplotlib
 
