What is the CPython behavior for this test case?

What is the CPython behavior for this test case?

I got curious, so I played around with this a bit.

import asyncio

async def task():
    raise Exception("task")

async def test():
    t = asyncio.create_task(task())
    await asyncio.sleep(0)
    await asyncio.sleep(0)
    await t

async def wait():
    await asyncio.sleep(2)
    print('done')

async def main():
    await asyncio.gather(test(), wait())

asyncio.run(main())

Results in:

Traceback (most recent call last):
  File "/home/david/work/junk/mp8929/test.py", line 19, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/david/work/junk/mp8929/test.py", line 17, in main
    await asyncio.gather(test(), wait())
  File "/home/david/work/junk/mp8929/test.py", line 10, in test
    await t
  File "/home/david/work/junk/mp8929/test.py", line 4, in task
    raise Exception("task")
Exception: task

If you comment out await t, then you get:

Task exception was never retrieved
future: <Task finished name='Task-4' coro=<task() done, defined at /home/david/work/junk/mp8929/test.py:3> exception=Exception('task')>
Traceback (most recent call last):
  File "/home/david/work/junk/mp8929/test.py", line 4, in task
    raise Exception("task")
Exception: task
done

Since CPython is reference counted, it knows the task (t) cannot be used any more, so it calls the loop exception handler right away (before done).

If you add global t as the first line of test() so that the task lives on past the end of the coroutine, you get:

done
Task exception was never retrieved
future: <Task finished name='Task-4' coro=<task() done, defined at /home/david/work/junk/mp8929/test.py:3> exception=Exception('task')>
Traceback (most recent call last):
  File "/home/david/work/junk/mp8929/test.py", line 4, in task
    raise Exception("task")
Exception: task

The loop exception handler is not called until the very end of the program.

What is the CPython behavior for this test case?

@dlech Thanks for reminding me, sorry forgot to include this above.

In this test case, CPython raises on the await (and never sends the exception to the loop handler).

My understanding is that only when a task becomes unreferenced does it get sent to the loop handler, and until then it's assumed that it will potentially be awaited.

If we consider the loop handler to be "informative", then that would be a good argument for alternative 2 above (i.e. retain the exception on the task).

My first instinct is to change the MicroPython behavior to be like the CPython case with the global t. By default, don't run the loop exception handler unless the task is still queued when the loop is stopped. Then we could add a Task.close() method that could be used to check for unawaited tasks, e.g. in a context manager. This could act as a replacement for the reference counting.

import asyncio

async def task():
    raise Exception("task")

async def test():
    async with asyncio.create_task(task()) as t:
        await asyncio.sleep(0)
        await asyncio.sleep(0)
        # await t <- not awaited task
    # <-- loop exception handler runs here

async def wait():
    await asyncio.sleep(2)
    print('done')

async def main():
    await asyncio.gather(test(), wait())

asyncio.run(main())

Unfortunately, this wouldn't be compatible with CPython, so probably not a practical solution.

If we consider the loop handler to be "informative", then that would be a good argument for alternative 2 above (i.e. retain the exception on the task).

I think not retaining the exception and raising when awaited is a bug. There could be cases where exceptions are expected.

t = asyncio.create_task(task())
...
try:
    await t
except SomeError:
    ...

In this case, SomeError is expected and handled but if the loop exception handler also runs before this, it just creates noise (which could be misleading if it says the exception was not handled, but it really was).

FYI, we incorporated this a while ago into CircuitPython, even though it is currently unmerged into MicroPython, as adafruit#7059.

Make the task retain its exception, and even after it has been delivered to the loop's exception handler, still have await raise the original exception.

This fix (option 2) was actually implemented as part of fixing gather(), see 2ecbad4

And the test from this PR was separately added in c3ca361

FYI, we incorporated this a while ago into CircuitPython

@dhalbert you probably want to revert this fix in CircuitPython.