Crash report

What happened?

Also falls under: RuntimeError: <_overlapped.Overlapped object at 0x<address>> still has pending operation at deallocation, the process may crash.

We have an application that embeds Python and needs to tick the event loop once (or more) per frame. Generally this is done with the event loop idiom: stop() / run_forever() (once #110771 is available we intend to use that but our workaround is keeping the event loop running between frames similar to #110773 ).

(On my machine this has an 80% chance of printing the above error within 60 seconds, higher the longer it runs)

import asyncio
import threading
import time

main_loop = asyncio.get_event_loop_policy().new_event_loop()
assert isinstance(main_loop, asyncio.ProactorEventLoop)

def threadMain():
    while True:
        main_loop.call_soon_threadsafe(lambda: None)
        time.sleep(0.01)

thr = threading.Thread(target=threadMain, daemon=True)
thr.start()

while True:
    main_loop.stop()
    main_loop.run_forever()

Initial diagnosis

Typically when the "still has pending operation" error message is printed, _overlapped_Overlapped_cancel_impl() will have been called which will call CancelIoEx() (returns TRUE).

When Overlapped_dealloc() is called and overlapped IO has not completed, CancelIoEx() is called again, returning FALSE with GLE=ERROR_NOT_FOUND (because the previous cancel succeeded). The existing code then releases the GIL and calls GetOverlappedResult() (wait = FALSE) which also returns FALSE with GLE=ERROR_IO_PENDING. Since Windows is not done with the OVERLAPPED struct the error message is printed and the memory is dangerously freed.

Changing wait = TRUE if either the _overlapped_Overlapped_cancel_impl() call to CancelIoEx() succeeds, or the one in Overlapped_dealloc() succeeds will result in a hang: GetOverlappedResult() will never return. This is because IO Completion Ports are in use and GetQueuedCompletionStatus() needs to be queried until it returns the OVERLAPPED in question.

CPython versions tested on:

3.10, 3.11, CPython main branch

Operating systems tested on:

Windows

Output from running 'python -VV' on the command line:

Python 3.13.0a4+ (heads/main:bb66600558, Mar 13 2024, 18:46:58) [MSC v.1938 64 bit (AMD64)]

Linked PRs

• gh-116773: Fix overlapped memory corruption issue #116774
• [3.12] gh-116773: Fix overlapped memory corruption crash (GH-116774) #117078
• [3.11] gh-116773: Fix overlapped memory corruption crash (GH-116774) #117077
• [3.10] gh-116773: Fix overlapped memory corruption crash (GH-116774) #117079
• [3.9] gh-116773: Fix overlapped memory corruption crash (GH-116774) #117080
• [3.8] gh-116773: Fix overlapped memory corruption crash (GH-116774) #117083