Skip to content

enum doctests are silently skipped when run with libregrtest #111181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sobolevn opened this issue Oct 22, 2023 · 2 comments
Closed

enum doctests are silently skipped when run with libregrtest #111181

sobolevn opened this issue Oct 22, 2023 · 2 comments
Assignees
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@sobolevn
Copy link
Member

sobolevn commented Oct 22, 2023

Bug report

test_enum has these lines:

def load_tests(loader, tests, ignore):
tests.addTests(doctest.DocTestSuite(enum))
if os.path.exists('Doc/library/enum.rst'):
tests.addTests(doctest.DocFileSuite(
'../../Doc/library/enum.rst',
optionflags=doctest.ELLIPSIS|doctest.NORMALIZE_WHITESPACE,
))
if os.path.exists('Doc/howto/enum.rst'):
tests.addTests(doctest.DocFileSuite(
'../../Doc/howto/enum.rst',
optionflags=doctest.ELLIPSIS|doctest.NORMALIZE_WHITESPACE,
))
return tests

They are problematic, because they are not executed when run as ./python.exe -m test test_enum.
Only when run with ./python.exe Lib/test/test_enum.py

Difference (on main branch):

» ./python.exe -m test test_enum
Using random seed: 1578294160
0:00:00 load avg: 1.98 Run 1 test sequentially
0:00:00 load avg: 1.98 [1/1] test_enum

== Tests result: SUCCESS ==

1 test OK.

Total duration: 856 ms
Total tests: run=1,060 skipped=4
Total test files: run=1/1
Result: SUCCESS

and:

» ./python.exe Lib/test/test_enum.py  
...............................................................................................................................................................................................................................s.s..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................s..........................................s........................................................................................................................................FF
======================================================================
FAIL: /Users/sobolev/Desktop/cpython/Lib/test/../../Doc/library/enum.rst
Doctest: enum.rst
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 2263, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for enum.rst
  File "/Users/sobolev/Desktop/cpython/Lib/test/../../Doc/library/enum.rst", line 0

----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython/Lib/test/../../Doc/library/enum.rst", line 301, in enum.rst
Failed example:
    dir(Weekday.SATURDAY)
Expected:
    ['__class__', '__doc__', '__eq__', '__hash__', '__module__', 'name', 'today', 'value']
Got:
    ['FRIDAY', 'MONDAY', 'SATURDAY', 'SUNDAY', 'THURSDAY', 'TUESDAY', 'WEDNESDAY', '__class__', '__doc__', '__eq__', '__hash__', '__module__', 'name', 'today', 'value']


======================================================================
FAIL: /Users/sobolev/Desktop/cpython/Lib/test/../../Doc/howto/enum.rst
Doctest: enum.rst
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 2263, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for enum.rst
  File "/Users/sobolev/Desktop/cpython/Lib/test/../../Doc/howto/enum.rst", line 0

----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython/Lib/test/../../Doc/howto/enum.rst", line 486, in enum.rst
Failed example:
    @dataclass
    class CreatureDataMixin:
        size: str
        legs: int
        tail: bool = field(repr=False, default=True)
Exception raised:
    Traceback (most recent call last):
      File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1374, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest enum.rst[64]>", line 1, in <module>
        @dataclass
         ^^^^^^^^^
    NameError: name 'dataclass' is not defined
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython/Lib/test/../../Doc/howto/enum.rst", line 492, in enum.rst
Failed example:
    class Creature(CreatureDataMixin, Enum):
        BEETLE = 'small', 6
        DOG = 'medium', 4
Exception raised:
    Traceback (most recent call last):
      File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1374, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest enum.rst[65]>", line 1, in <module>
        class Creature(CreatureDataMixin, Enum):
                       ^^^^^^^^^^^^^^^^^
    NameError: name 'CreatureDataMixin' is not defined
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython/Lib/test/../../Doc/howto/enum.rst", line 496, in enum.rst
Failed example:
    Creature.DOG
Exception raised:
    Traceback (most recent call last):
      File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1374, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest enum.rst[66]>", line 1, in <module>
        Creature.DOG
        ^^^^^^^^
    NameError: name 'Creature' is not defined
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython/Lib/test/../../Doc/howto/enum.rst", line 530, in enum.rst
Failed example:
    class MyEnum(Enum):
        __reduce_ex__ = enum.pickle_by_enum_name
Exception raised:
    Traceback (most recent call last):
      File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1374, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest enum.rst[70]>", line 1, in <module>
        class MyEnum(Enum):
      File "<doctest enum.rst[70]>", line 2, in MyEnum
        __reduce_ex__ = enum.pickle_by_enum_name
                        ^^^^
    NameError: name 'enum' is not defined. Did you mean: 'Enum'? Or did you forget to import 'enum'?
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython/Lib/test/../../Doc/howto/enum.rst", line 773, in enum.rst
Failed example:
    Perm.X | 8
Expected:
    9
Got:
    <Perm.X|8: 9>
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython/Lib/test/../../Doc/howto/enum.rst", line 1431, in enum.rst
Failed example:
    class Color(DuplicateFreeEnum):
        RED = 1
        GREEN = 2
        BLUE = 3
        GRENE = 2
Expected:
    Traceback (most recent call last):
    ...
    ValueError: aliases not allowed in DuplicateFreeEnum:  'GRENE' --> 'GREEN'
Got:
    Traceback (most recent call last):
      File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1374, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest enum.rst[162]>", line 1, in <module>
        class Color(DuplicateFreeEnum):
      File "/Users/sobolev/Desktop/cpython/Lib/enum.py", line 570, in __new__
        enum_class = super().__new__(metacls, cls, bases, classdict, **kwds)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/sobolev/Desktop/cpython/Lib/enum.py", line 278, in __set_name__
        enum_member.__init__(*args)
      File "<doctest enum.rst[161]>", line 7, in __init__
        raise ValueError(
    ValueError: aliases not allowed in DuplicateFreeEnum:  'GRENE' --> 'GREEN'
    Error calling __set_name__ on '_proto_member' instance 'GRENE' in 'Color'


----------------------------------------------------------------------
Ran 1062 tests in 0.876s

FAILED (failures=2, skipped=4)

Why? Because of

if os.path.exists('Doc/library/enum.rst'):
checks. This file does not exist when executed via -m test, here what os.getcwd() shows for -m test: .../cpython/build/test_python_worker_45291æ

But, with ./python.exe Lib/test/test_enum.py it is: /Users/sobolev/Desktop/cpython

I've made a PR #111180 with fixes that will satisfy doctest, but I am not sure they are correct.

CC @ethanfurman

Linked PRs

@sobolevn sobolevn added type-bug An unexpected behavior, bug, or error stdlib Python modules in the Lib dir labels Oct 22, 2023
@sobolevn sobolevn self-assigned this Oct 22, 2023
@sobolevn sobolevn changed the title enum doctests are siltently skipped when runner with libregrtest enum doctests are siltently skipped when run with libregrtest Oct 22, 2023
@AlexWaygood AlexWaygood changed the title enum doctests are siltently skipped when run with libregrtest enum doctests are silently skipped when run with libregrtest Oct 22, 2023
@chgnrdv
Copy link
Contributor

chgnrdv commented Oct 22, 2023

This is apparently a duplicate of related to #104235 (#104236), but I would prefer to close them because I find your fix more accurate.

See also #107957(#107958) for library/enum.rst docs changes.

ethanfurman added a commit that referenced this issue Oct 30, 2023
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 30, 2023
(cherry picked from commit c4dc5a6)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
ethanfurman added a commit that referenced this issue Oct 31, 2023
gh-111181: Fix enum doctests (GH-111180)
(cherry picked from commit c4dc5a6)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
ethanfurman added a commit to ethanfurman/cpython that referenced this issue Nov 1, 2023
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
(cherry picked from commit c4dc5a6)
FullteaR pushed a commit to FullteaR/cpython that referenced this issue Nov 3, 2023
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
ethanfurman added a commit that referenced this issue Nov 3, 2023
gh-111181: Fix enum doctests (GH-111180)

Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
(cherry picked from commit c4dc5a6)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
@hugovk
Copy link
Member

hugovk commented Nov 9, 2023

Thanks for the PRs!

@hugovk hugovk closed this as completed Nov 9, 2023
aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

3 participants