asyncio.run unnecessarily calls the repr of the task result twice since Python 3.11 #112559

yilei · 2023-11-30T18:08:00Z

Bug report

Bug description:

Given the following code:

import asyncio
import time

class Foo:
    def __repr__(self):
        time.sleep(1)
        print('i am a repr, i should not be called. ')
        return '<Foo>'


async def get_foo():
    return Foo()

asyncio.run(get_foo())
print('Done')

Output:

$ python t.py
i am a repr, i should not be called.
i am a repr, i should not be called.
Done

This was caused by the new SIGINT handler installed by asyncio.run here: f08a191

Upon investigation, changing the code with:

import asyncio
import time

class Foo:
    def __repr__(self):
        time.sleep(1)
        print('i am a repr, i should not be called. ')
        raise BaseException('where is this called???????')
        return '<Foo>'


async def get_foo():
    return Foo()

asyncio.run(get_foo())
print('Done')

It shows:

i am a repr, i should not be called.
Traceback (most recent call last):
  File "t.py", line 15, in <module>
    asyncio.run(get_foo())
  File "lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "lib/python3.12/asyncio/runners.py", line 127, in run
    and signal.getsignal(signal.SIGINT) is sigint_handler
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/signal.py", line 63, in getsignal
    return _int_to_enum(handler, Handlers)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/signal.py", line 29, in _int_to_enum
    return enum_klass(value)
           ^^^^^^^^^^^^^^^^^
  File "lib/python3.12/enum.py", line 740, in __call__
    return cls.__new__(cls, value)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/enum.py", line 1152, in __new__
    ve_exc = ValueError("%r is not a valid %s" % (value, cls.__qualname__))
                                                  ^^^^^
  File "lib/python3.12/reprlib.py", line 21, in wrapper
    result = user_function(self)
             ^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/asyncio/base_tasks.py", line 30, in _task_repr
    info = ' '.join(_task_repr_info(task))
                    ^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/asyncio/base_tasks.py", line 10, in _task_repr_info
    info = base_futures._future_repr_info(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/asyncio/base_futures.py", line 54, in _future_repr_info
    result = reprlib.repr(future._result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/reprlib.py", line 58, in repr
    return self.repr1(x, self.maxlevel)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/reprlib.py", line 68, in repr1
    return self.repr_instance(x, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/reprlib.py", line 170, in repr_instance
    s = builtins.repr(x)
        ^^^^^^^^^^^^^^^^
  File "t.py", line 8, in __repr__
    raise BaseException('where is this called???????')
BaseException: where is this called???????

This looks like a series unfortunate events, running on 3.12.0:

signal.getsignal tries to convert the handler function to enum in case this is part of Handlers:

cpython/Lib/signal.py

Line 29 in 0fb18b0

return enum_klass(value)
enum raises a ValueError with the repr of the handler function:

cpython/Lib/enum.py

Line 1152 in 0fb18b0

ve_exc = ValueError("%r is not a valid %s" % (value, cls.__qualname__))
the handler asyncio.run installs uses a functools.partial, it's repr will include the repr of the task:

cpython/Lib/asyncio/runners.py

Line 105 in 0fb18b0

sigint_handler = functools.partial(self._on_sigint, main_task=task)

when the repr is actually called at the end (one in signal.getsignal, the other in signal.signal), the repr of the asyncio task will include the repr of the result:

cpython/Lib/asyncio/runners.py

Lines 127 to 129 in 0fb18b0

    
               and signal.getsignal(signal.SIGINT) is sigint_handler 
        
           ): 
        
               signal.signal(signal.SIGINT, signal.default_int_handler)

While one can argument calling __repr__ shouldn't cause issues, but I think we could avoid them in the signal._int_to_enum function completely, by only trying to convert to enum when it's an integer:

def _int_to_enum(value, enum_klass):
    """Convert a numeric value to an IntEnum member.
    If it's not a known member, return the numeric value itself.
    """
    if not isinstance(value, int):
        return value
    try:
        return enum_klass(value)
    except ValueError:
        return value

This should be more efficient on its own anyway.

This function's doc is also inaccurate, since it also accepts non integers (usually a callable).

Am I missing something?

CPython versions tested on:

3.11, 3.12

Operating systems tested on:

Linux

Linked PRs

The text was updated successfully, but these errors were encountered:

… of a coroutine as integer while fetching sigint handler. This makes the test materialize the whole tensor in memory. This changes the test co-routine to return nothing to avoid triggering this bug. python/cpython#112559 PiperOrigin-RevId: 586545760

… of a coroutine as integer while fetching sigint handler. This makes the test materialize the whole tensor in memory. This changes the test co-routine to return nothing to avoid triggering this bug. python/cpython#112559 PiperOrigin-RevId: 586756112

gvanrossum · 2023-12-01T03:44:40Z

Thanks for the very thorough bug report. I am asking @kumaraditya303 to help out. Maybe @ethanfurman can also shed some light on this, since it looks like you are proposing a fix to the enum module.

yilei · 2023-12-01T03:47:11Z

Thanks for the response! I'm proposing a change in signal.py module's private function here:

cpython/Lib/signal.py

Line 24 in 5b06299

def _int_to_enum(value, enum_klass):

gvanrossum · 2023-12-13T02:58:28Z

@yilei Could you just submit a PR for that?

…in signal.py.

…nal.py (#113040)

…in signal.py (pythonGH-113040) (cherry picked from commit 050783c) Co-authored-by: Yilei Yang <yileiyang@google.com>

… in signal.py (GH-113040) (#113444) gh-112559: Avoid unnecessary conversion attempts to enum_klass in signal.py (GH-113040) (cherry picked from commit 050783c) Co-authored-by: Yilei Yang <yileiyang@google.com>

… in signal.py (GH-113040) (#113443) (cherry picked from commit 050783c) Co-authored-by: Yilei Yang <yileiyang@google.com>

…in signal.py (python#113040)

yilei added the type-bug An unexpected behavior, bug, or error label Nov 30, 2023

Eclips4 added the topic-asyncio label Nov 30, 2023

github-project-automation bot added this to asyncio Nov 30, 2023

github-project-automation bot moved this to Todo in asyncio Nov 30, 2023

AlexWaygood added stdlib Python modules in the Lib dir 3.11 only security fixes 3.12 only security fixes 3.13 bugs and security fixes labels Nov 30, 2023

yilei added a commit to yilei/cpython that referenced this issue Dec 13, 2023

pythongh-112559: Avoid unnecessary conversion attempts to enum_klass …

2afdd90

…in signal.py.

yilei mentioned this issue Dec 13, 2023

gh-112559: Avoid unnecessary conversion attempts to enum_klass in signal.py. #113040

Merged

gvanrossum pushed a commit that referenced this issue Dec 24, 2023

gh-112559: Avoid unnecessary conversion attempts to enum_klass in sig…

050783c

…nal.py (#113040)

This was referenced Dec 24, 2023

[3.12] gh-112559: Avoid unnecessary conversion attempts to enum_klass in signal.py (GH-113040) #113443

Merged

[3.11] gh-112559: Avoid unnecessary conversion attempts to enum_klass in signal.py (GH-113040) #113444

Merged

gvanrossum pushed a commit that referenced this issue Dec 24, 2023

[3.12] gh-112559: Avoid unnecessary conversion attempts to enum_klass…

7f2ebc8

… in signal.py (GH-113040) (#113443) (cherry picked from commit 050783c) Co-authored-by: Yilei Yang <yileiyang@google.com>

gvanrossum closed this as completed Dec 24, 2023

github-project-automation bot moved this from Todo to Done in asyncio Dec 24, 2023

ryan-duve pushed a commit to ryan-duve/cpython that referenced this issue Dec 26, 2023

pythongh-112559: Avoid unnecessary conversion attempts to enum_klass …

2e10d7e

…in signal.py (python#113040)

kulikjak pushed a commit to kulikjak/cpython that referenced this issue Jan 22, 2024

pythongh-112559: Avoid unnecessary conversion attempts to enum_klass …

c64b925

…in signal.py (python#113040)

aisk pushed a commit to aisk/cpython that referenced this issue Feb 11, 2024

pythongh-112559: Avoid unnecessary conversion attempts to enum_klass …

4e623af

…in signal.py (python#113040)

Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024

pythongh-112559: Avoid unnecessary conversion attempts to enum_klass …

3fdefe9

…in signal.py (python#113040)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

asyncio.run unnecessarily calls the repr of the task result twice since Python 3.11 #112559

asyncio.run unnecessarily calls the repr of the task result twice since Python 3.11 #112559

yilei commented Nov 30, 2023 •

edited by bedevere-app bot

Loading

gvanrossum commented Dec 1, 2023

yilei commented Dec 1, 2023

gvanrossum commented Dec 13, 2023

asyncio.run unnecessarily calls the repr of the task result twice since Python 3.11 #112559

asyncio.run unnecessarily calls the repr of the task result twice since Python 3.11 #112559

Comments

yilei commented Nov 30, 2023 • edited by bedevere-app bot Loading

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

gvanrossum commented Dec 1, 2023

yilei commented Dec 1, 2023

gvanrossum commented Dec 13, 2023

yilei commented Nov 30, 2023 •

edited by bedevere-app bot

Loading