Skip to content

gh-43657: Add support for custom test case and runner in both DocTestSuite and DocFileSuite #133203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 28 additions & 2 deletions Doc/library/doctest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1038,7 +1038,7 @@ There are two main functions for creating :class:`unittest.TestSuite` instances
from text files and modules with doctests:


.. function:: DocFileSuite(*paths, module_relative=True, package=None, setUp=None, tearDown=None, globs=None, optionflags=0, parser=DocTestParser(), encoding=None)
.. function:: DocFileSuite(*paths, module_relative=True, package=None, setUp=None, tearDown=None, globs=None, optionflags=0, parser=DocTestParser(), encoding=None, test_case=DocFileCase, runner=DocTestRunner)

Convert doctest tests from one or more text files to a
:class:`unittest.TestSuite`.
Expand Down Expand Up @@ -1102,11 +1102,24 @@ from text files and modules with doctests:
Optional argument *encoding* specifies an encoding that should be used to
convert the file to unicode.

Optional argument *test_case* specifies the :class:`!DocFileCase` class (or a
subclass) that should be used to create test cases. By default, :class:`!DocFileCase`
is used. This allows for custom test case classes that can add additional behavior
or attributes to the test cases.

Optional argument *runner* specifies the :class:`DocTestRunner` class (or a
subclass) that should be used to run the tests. By default, :class:`DocTestRunner`
is used.

The global ``__file__`` is added to the globals provided to doctests loaded
from a text file using :func:`DocFileSuite`.

.. versionchanged:: 3.13
Added *test_case* and *runner* parameters to support user specified test case
and runner in DocFileSuite.


.. function:: DocTestSuite(module=None, globs=None, extraglobs=None, test_finder=None, setUp=None, tearDown=None, optionflags=0, checker=None)
.. function:: DocTestSuite(module=None, globs=None, extraglobs=None, test_finder=None, setUp=None, tearDown=None, optionflags=0, checker=None, test_case=DocTestCase, runner=DocTestRunner)

Convert doctest tests for a module to a :class:`unittest.TestSuite`.

Expand Down Expand Up @@ -1134,12 +1147,25 @@ from text files and modules with doctests:
Optional arguments *setUp*, *tearDown*, and *optionflags* are the same as for
function :func:`DocFileSuite` above.

Optional argument *test_case* specifies the :class:`!DocTestCase` class (or a
subclass) that should be used to create test cases. By default, :class:`!DocTestCase`
is used. This allows for custom test case classes that can add additional behavior
or attributes to the test cases.

Optional argument *runner* specifies the :class:`DocTestRunner` class (or a
subclass) that should be used to run the tests. By default, :class:`DocTestRunner`
is used.

This function uses the same search technique as :func:`testmod`.

.. versionchanged:: 3.5
:func:`DocTestSuite` returns an empty :class:`unittest.TestSuite` if *module*
contains no docstrings instead of raising :exc:`ValueError`.

.. versionchanged:: 3.13
Added *runner* and *test_case* parameters to support custom test runners
and test case classes in DocTestSuite.

.. exception:: failureException

When doctests which have been converted to unit tests by :func:`DocFileSuite`
Expand Down
37 changes: 30 additions & 7 deletions Lib/doctest.py
Original file line number Diff line number Diff line change
Expand Up @@ -2275,14 +2275,15 @@ def set_unittest_reportflags(flags):
class DocTestCase(unittest.TestCase):

def __init__(self, test, optionflags=0, setUp=None, tearDown=None,
checker=None):
checker=None, runner=DocTestRunner):

unittest.TestCase.__init__(self)
self._dt_optionflags = optionflags
self._dt_checker = checker
self._dt_test = test
self._dt_setUp = setUp
self._dt_tearDown = tearDown
self._dt_runner = runner

def setUp(self):
test = self._dt_test
Expand Down Expand Up @@ -2312,7 +2313,7 @@ def runTest(self):
# so add the default reporting flags
optionflags |= _unittest_reportflags

runner = DocTestRunner(optionflags=optionflags,
runner = self._dt_runner(optionflags=optionflags,
checker=self._dt_checker, verbose=False)

try:
Expand Down Expand Up @@ -2460,7 +2461,7 @@ def _removeTestAtIndex(self, index):


def DocTestSuite(module=None, globs=None, extraglobs=None, test_finder=None,
**options):
test_case=DocTestCase, runner=DocTestRunner, **options):
"""
Convert doctest tests for a module to a unittest test suite.

Expand Down Expand Up @@ -2494,6 +2495,12 @@ def DocTestSuite(module=None, globs=None, extraglobs=None, test_finder=None,

optionflags
A set of doctest option flags expressed as an integer.

test_case
A custom optional DocTestCase class to use for test cases.

runner
A custom optional DocTestRunner class to use for running tests.
"""

if test_finder is None:
Expand All @@ -2519,7 +2526,7 @@ def DocTestSuite(module=None, globs=None, extraglobs=None, test_finder=None,
if filename[-4:] == ".pyc":
filename = filename[:-1]
test.filename = filename
suite.addTest(DocTestCase(test, **options))
suite.addTest(test_case(test, runner=runner, **options))

return suite

Expand All @@ -2538,7 +2545,8 @@ def format_failure(self, err):

def DocFileTest(path, module_relative=True, package=None,
globs=None, parser=DocTestParser(),
encoding=None, **options):
encoding=None, test_case=DocFileCase,
runner=DocTestRunner, **options):
if globs is None:
globs = {}
else:
Expand All @@ -2560,7 +2568,7 @@ def DocFileTest(path, module_relative=True, package=None,

# Convert it to a test, and wrap it in a DocFileCase.
test = parser.get_doctest(doc, globs, name, path, 0)
return DocFileCase(test, **options)
return test_case(test, runner=runner, **options)

def DocFileSuite(*paths, **kw):
"""A unittest suite for one or more doctest files.
Expand Down Expand Up @@ -2617,6 +2625,12 @@ def DocFileSuite(*paths, **kw):

encoding
An encoding that will be used to convert the files to unicode.

test_case
A custom DocFileCase subclass to use for the tests.

runner
A custom DocTestRunner subclass to use for running the tests.
"""
suite = _DocTestSuite()

Expand All @@ -2626,8 +2640,17 @@ def DocFileSuite(*paths, **kw):
if kw.get('module_relative', True):
kw['package'] = _normalize_module(kw.get('package'))

test_case = kw.pop('test_case', DocFileCase)
runner = kw.pop('runner', DocTestRunner)

for path in paths:
suite.addTest(DocFileTest(path, **kw))
test_instance = DocFileTest(
path=path,
test_case=test_case,
runner=runner,
**kw
)
suite.addTest(test_instance)

return suite

Expand Down
62 changes: 62 additions & 0 deletions Lib/test/test_doctest/test_doctest.py
Original file line number Diff line number Diff line change
Expand Up @@ -2383,6 +2383,36 @@ def test_DocTestSuite():
modified the test globals, which are a copy of the
sample_doctest module dictionary. The test globals are
automatically cleared for us after a test.

We can also provide a custom test case class:

>>> class CustomDocTestCase(doctest.DocTestCase):
... def __init__(self, test, **options):
... super().__init__(test, **options)
... self.custom_attr = "custom_value"
... def runTest(self):
... self.assertEqual(self.custom_attr, "custom_value")
... super().runTest()

>>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest',
... test_case=CustomDocTestCase)
>>> result = suite.run(unittest.TestResult())
>>> result
<unittest.result.TestResult run=9 errors=0 failures=4>

We can also provide both a custom test case class and a custom runner class:

>>> class CustomDocTestRunner(doctest.DocTestRunner):
... def __init__(self, **options):
... super().__init__(**options)
... self.custom_attr = "custom_runner"
>>> suite = doctest.DocTestSuite('test.test_doctest.sample_doctest',
... test_case=CustomDocTestCase,
... runner=CustomDocTestRunner)
>>> result = suite.run(unittest.TestResult())
>>> result
<unittest.result.TestResult run=9 errors=0 failures=4>

"""

def test_DocFileSuite():
Expand Down Expand Up @@ -2543,6 +2573,38 @@ def test_DocFileSuite():
>>> suite.run(unittest.TestResult())
<unittest.result.TestResult run=3 errors=0 failures=2>

We can also provide a custom test case class:

>>> class CustomDocTestCase(doctest.DocFileCase):
... def __init__(self, test, **options):
... super().__init__(test, **options)
... self.custom_attr = "custom_value"
... def runTest(self):
... self.assertEqual(self.custom_attr, "custom_value")
... super().runTest()

>>> suite = doctest.DocFileSuite('test_doctest.txt',
... globs={'favorite_color': 'blue'},
... test_case=CustomDocTestCase)
>>> result = suite.run(unittest.TestResult())
>>> result
<unittest.result.TestResult run=1 errors=0 failures=0>

We can also provide both a custom test case class and a custom runner class:

>>> class CustomDocTestRunner(doctest.DocTestRunner):
... def __init__(self, **options):
... super().__init__(**options)
... self.custom_attr = "custom_runner"

>>> suite = doctest.DocFileSuite('test_doctest.txt',
... globs={'favorite_color': 'blue'},
... test_case=CustomDocTestCase,
... runner=CustomDocTestRunner)
>>> result = suite.run(unittest.TestResult())
>>> result
<unittest.result.TestResult run=1 errors=0 failures=0>

"""

def test_trailing_space_in_test():
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add support for custom test case and runner classes in :func:`doctest.DocTestSuite` and :func:`doctest.DocFileSuite`. This allows users to extend doctest functionality by providing their own test case and runner implementations through the new ``test_case`` and ``runner`` parameters.
Loading