Skip to content

Every input in asyncio REPL (wrongly) runs in a separate PEP 567 context #124593

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bswck-hai opened this issue Sep 26, 2024 · 1 comment
Closed
Labels
type-bug An unexpected behavior, bug, or error

Comments

@bswck-hai
Copy link

bswck-hai commented Sep 26, 2024

Bug report

Bug description:

The bug is twofold, affects the asyncio REPL and refers to PEP 567 contexts and context variables.

  1. Non-async inputs don't run in the same context.

    This is how we can use the standard Python REPL to demonstrate context variables (without asyncio tasks):

    Python 3.14.0a0 (heads/main-dirty:8447c933da3, Sep 25 2024, 15:45:56) [GCC 11.4.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import contextvars
    >>> var = contextvars.ContextVar("var")
    >>> var.set("ok")
    <Token var=<ContextVar name='var' at ...> at ...>
    >>> var.get()
    'ok'
    

    I'd expect the asyncio REPL to be a slightly better tool for presenting them though. However, I ran into a problem with running the same sequence of instructions in that REPL, because it implicitly copies the context for every input and then runs the instructions in that copied context:

    asyncio REPL 3.14.0a0 (heads/main-dirty:8447c933da3, Sep 25 2024, 15:45:56) [GCC 11.4.0] on linux
    Use "await" directly instead of "asyncio.run()".
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import asyncio
    >>> import contextvars
    >>> var = contextvars.ContextVar("var")
    >>> var.set("ok")
    <Token var=<ContextVar name='var' at ...> at ...>
    >>> var.get()
    Traceback (most recent call last):
      File ".../cpython/Lib/concurrent/futures/_base.py", line 448, in result
        return self.__get_result()
               ~~~~~~~~~~~~~~~~~^^
      File ".../cpython/Lib/concurrent/futures/_base.py", line 393, in __get_result
        raise self._exception
      File ".../cpython/Lib/asyncio/__main__.py", line 40, in callback
        coro = func()
      File "<python-input-3>", line 1, in <module>
        var.get()
        ~~~~~~~^^
    LookupError: <ContextVar name='var' at ...>
    

    I assume it to just be a leftover before the context parameter for asyncio routines became a thing in 3.11.

  2. Async inputs don't run in the same context.
    Similar to the first part, but this applies to every coroutine being run in the REPL.

    By standard, a simple coroutine does not run in a separate context (in contrast to tasks that document their handling of context). Consider the following script:

    import asyncio
    from contextvars import ContextVar
    
    var = ContextVar("var", default="unset")
    
    async def setvar() -> None:
        var.set("set")
    
    async def main() -> None:
        await setvar()  # runs in *this* context
        print(var.get())  # gets us "set", not "unset"
    
    asyncio.run(main())
    print(var.get())  # coro passed to asyncio.run() runs in a separate context, which gets us "unset" here

    In the REPL, I'd expect that we're inside the "global" context of asyncio.run()—what I mean by this is that the context we work in (for the entire REPL session) is the same one that is reused to run every asynchronous input, because we're in the same "main" task, and the point of asyncio REPL is to be able to omit it. While it is expected that every asynchronous input with top-level await statements becomes a task for the current event loop internally, I would imagine that the behavior resembles what I'd get by merging the inputs together and putting them inside a async def main() function in a Python script.
    For that reason, I consider the following a part of the bug:

    asyncio REPL 3.14.0a0 (heads/main-dirty:8447c933da3, Sep 25 2024, 15:45:56) [GCC 11.4.0] on linux
    Use "await" directly instead of "asyncio.run()".
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import asyncio
    >>> import contextvars
    >>> var = contextvars.ContextVar("var")
    >>> async def setvar():
    ...     var.set("ok")
    ...     
    >>> await setvar()
    >>> var.get()
    Traceback (most recent call last):
      File ".../cpython/Lib/concurrent/futures/_base.py", line 448, in result
        return self.__get_result()
               ~~~~~~~~~~~~~~~~~^^
      File ".../cpython/Lib/concurrent/futures/_base.py", line 393, in __get_result
        raise self._exception
      File ".../cpython/Lib/asyncio/__main__.py", line 40, in callback
        coro = func()
      File "<python-input-4>", line 1, in <module>
        var.get()
        ~~~~~~~^^
    LookupError: <ContextVar name='var' at 0x7f7c6e4e6390>
    

    I'd expect every asynchronous input to not run in an implicitly copied context, because that's what feels like if we were writing a huge async def main() function—every await does not create a task, it just awaits the coroutine (the execution of which is managed by the event loop).

Interestingly, IPython (which works asynchronously under the hood too) only runs to the second part of the bug.

Thanks to the already mentioned GH-91150, this is very simple to manage and fix. Expect a PR soon.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

@bswck-hai bswck-hai added the type-bug An unexpected behavior, bug, or error label Sep 26, 2024
@bswck-hai
Copy link
Author

Oops, wrong account.

@bswck-hai bswck-hai closed this as not planned Won't fix, can't repro, duplicate, stale Sep 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

1 participant