As noted in bpo-10224, python 3.x documentation is not being built with python 3.x yet, so a simple cd Doc; make doctest does not work. However, hg trunk version os sphinx works with python 3.x and can be used to validate examples in ReST documentation.

hg trunk version [of] sphinx works with python 3.x and can be used to
validate examples in ReST documentation.

How do you replace “make doctest”?

On Thu, Oct 28, 2010 at 10:20 PM, Éric Araujo <report@bugs.python.org> wrote:

How do you replace “make doctest”?

$ sphinx-build -b html -d Doc/build3/doctrees -D latex_paper_size=
Doc  Doc/build3/html

but I had to run 2to3 on Doc/conf.py and Doc/tools first.

I started with 2.7 branch because some of the issues are the same there, but the tools work better at the moment. I a posting a work-in-progress patch to solicit early feedback.

I am attaching a new patch which fixes all but two doctest examples.

I suspect that the remaining failures are due to a bug in sphinx. (The examples are executed even though marked up with ::).

I started porting the patch to 3.x and it looks like I've uncovered another bug in sphinx:

$ sphinx-build -b doctest -d build/doctrees  . build/doctest library/traceback.rst
..
**********************************************************************
File "library/traceback.rst", line 327, in default
..

Differences (ndiff with -expected +actual):
*** print_tb:
- File "<doctest...>", line 10, in <module>
? ^^^
+ File "<doctest default[0]>", line 10, in <module>
? ^^^^^^^^^^^
lumberjack()

(I added +REPORT_NDIFF to testoutput options for clarity.)

I even tried adding +ELLIPSIS to options, but this did not help. The same example works in 2.7.

As I understand this, your patch would fix numerous errors in the docs, but has not been applied. It would be nice to get the fixes into 3.2. What sort of feedback do you want? Proofreading of text? Recheck of doctest?

Does sphinx suppress doctest comments like
#doctest: +IGNORE_EXCEPTION_DETAIL
?

doctest.testfile("C:/programs/PyDev/py32/Doc/howto/sorting.rst", module_relative = False) fails 23 of 37 attempts, so improvement is needed ;-).

On Sun, Jan 9, 2011 at 3:52 PM, Terry J. Reedy <report@bugs.python.org> wrote:

.. It would be nice to get the fixes into 3.2.

Yes, it would be nice, but I don't have a 3.2 patch (yet). As you can
see from my last comment, I ran into a bug in the alpha version of
sphinx that I used to validate py3k docs.

What sort of feedback do you want? Proofreading of text? Recheck of doctest?

All of the above. Some of the fixes may be at the expense of
readability. I would appreciate feedback in the instances when human
and computer reader's interests are in conflict. Some examples are
"fixed" by excluding them from being checked. Better ideas are
welcome.

Does sphinx suppress doctest comments like
    #doctest: +IGNORE_EXCEPTION_DETAIL
?

Yes.

doctest.testfile("C:/programs/PyDev/py32/Doc/howto/sorting.rst", module_relative = False)
fails 23 of 37 attempts, so improvement is needed ;-).

You cannot run documentation examples with a plain doctest. You have
to use sphinx-build.

Doctest runs on .rst files, which are plain text files, finds and reports errors, and reports no errors when the errors are fixed. See bpo-10875 where is was very helpful. So your last comment puzzles me.

In any case, your patch is too big to digest at once. I extracted the part for howto/sorting.rst and tried to apply to 3.2, but the file has changed too much (at least for TortoiseSVN). So I pasted the '... 's (and adjusted spacing) where needed, and did the two other changes you had. This left just two failures -- for the 2.7 code using 'cmp=xxx' args. With '# doctest: +SKIP' added twice, doctest passes. Patch commited for 3.2 as r87946.

instances when human and computer reader's interests are in conflict.
What is one (or more) that you were thinking of?

On Tue, Jan 11, 2011 at 6:10 PM, Terry J. Reedy <report@bugs.python.org> wrote:
..

Doctest runs on .rst files, which are plain text files, finds and reports errors, and reports no errors when the errors are fixed.
See bpo-10875 where is was very helpful. So your last comment puzzles me.

Sphinx doctest runner supports doctest fixtures that plain doctest
does not. Some of the examples in the docs rely on these.

See http://sphinx.pocoo.org/ext/doctest.html .

Terry, I'm unhappy with the changes to the sorting how-to. IMO, it was a not a net win (transforming code that already ran fine in something doctest would swallow). The code snippets now have the visual clutter of the ">>>" and "..." PS1 and PS2 prompts and they can no longer be readily cut and pasted into the interpreter so that people can experiment with them.

There are two reasons I forward ported the changes.

1. Without running doctest on doc examples, they sometimes have errors either originally, after patches to the doc, or after patches to Python. On other issues, I found 4 errors in the json doc (probably original; my fixes were augmented and committed by GB, I believe), and several in the re howto (due to non-update after re changes). There have been other error reports on the tracker and I presume more (other than missing '...') shown in AB's patch. (I just started with the first file changed in the patch.) So I thought it pretty well settled that getting doc examples correct (and test-ready) is be a good idea.

2. Examples with output always start with '>>> ' to differentiate input from output. I thought it pretty standard that continuation lines start with '... '. This is true of all applicable examples in re howto, module chapters for difflib, json, bisect, and 4 of 5 applicable examples in the Built-in Types chapter. Other chapters for binhex, itertools, and tkinter have no multiline interactive examples, so provide no precedent either way.

What is the alternative? The current sorting howto is not consistent:

For some examples, '... ' is just deleted:
>>> student_tuples = [
    ('john', 'A', 15),
    ('jane', 'B', 12),
    ('dave', 'B', 10),
]

For others, '... ' is replaced with '    ':
>>> class Student:
        def __init__(self, name, grade, age):
            self.name = name
            self.grade = grade
            self.age = age

The cmp_to_key function def is a special case: there is no output in the same box with the def and so (normally) should not have prompts. The only reason I followed AB's patch here and added them is because the definition is used in the next box. If the next example were changed for 3.2 to use the new-in-3.2 functools.cmp_to_key(), then the def would not be needed. (I already planned to suggest thisOr the followup example could just be marked SKIP. With either change, I would remove the prompts added to this function, which are the ones I presume bother you the most.

Summary: according to my current (and subject-to-update ;-) understanding of precedent and policy, the changes other than for cmp_to_key seem correct. The rationale for cmp_to_key changes could easily be negated, in which case they should be reverted.

I will put my thoughts on cut-and-paste in a separate message.

You're welcome to discuss this with me on IRC at some point. For now, the important thing is that I put a good deal of time and effort working on the sorting howto and I like it as it stands (your notions of precedent or consistency not withstanding).

On Tue, Jan 11, 2011 at 10:03 PM, Terry J. Reedy <report@bugs.python.org> wrote:
..

What is the alternative?

I am attaching an alternative patch for sorting howto. You can run the
doctests in it as follows:

$ python2.7 tools/sphinx-build.py -b doctest -d build/doctrees .
build/doctest howto/sorting.rst
...

Doctest summary
===============
34 tests
0 failures in tests
0 failures in setup code
build succeeded.

Unfortunately, Sphinx does not support Ramond's preferred style, but
my patch comes close.

I have updated Sphinx and rerun py3k doctests. It looks like the newest Sphinx does not check examples unless the file has a .. testsetup:: directive somewhere. This is nice because it focuses on the files that are clearly intended to be doctest-checked. Of course, there are many perfectly good code examples that are now skipped, but this can be addressed in the next step.

I am attaching a patch that makes the following command succeed in py3k Doc directory:

$ sphinx-build --version
Sphinx v1.1pre
..
$ sphinx-build -b doctest -d build/doctrees . build/doctest
..
Doctest summary

===============
581 tests
0 failures ..

Most of the changes in bpo-10225-py3k.diff are non-controvercial with the exception of code fixes in collections and difflib.

In collections, I removed trailing whitespace from generated namedtuple code and in difflib I changed get_close_matches() return value from map object to a list. I would appreciate comments on these two changes.

Committed the simple fixes in r87957. Updated the patch to include only the remaining changes.

These are fine except for the changes to collections.py. Please leave the deployed code for named tuple as-is. Doctest may have issues with trailing whitespace, but that is doctest's problem, not named tuple's.

Please leave the deployed code for named tuple as-is. Doctest may
have issues with trailing whitespace, but that is doctest's problem,
not named tuple's.

I am curious, what was the reason to add trailing whitespace in the named tuple template?

I am curious, what was the reason to add
trailing whitespace in the named tuple template?

To make it hard to doctest ;-)

I had a thought that it made the template more readable, but the better solution was to just use real newlines instead of '\n'. The template has been considerably beautified for python 3.3, but I don't want to go back in time and muck with stable code just to accommodate adding a doctest in old docs.

Hi, I would like to revive this issue and have added a review comment to bpo-10225-py3k.diff.

Otherwise, I have reviewed the changes in both diffs and think they are good to go.

Andy

For whatever difference it makes, the 3.x docs now *are* being built with 3.x.

difflib.rst only needed a couple of imports to change 34 errors to 0. I also fixed 1 word error. Leave other modules as is and close.

I think that we've missed executing these doctests in our test suite. I will send a PR :)