gh-132493: Avoid eager import of annotationlib in typing (again) #132596
Conversation
pythongh-132494 made typing.py eagerly import annotationlib again because typing contains several protocols. Avoid this by determining annotations lazily. This should also make protocol creation faster: Unpatched: $ ./python.exe -m timeit -s 'from typing import Protocol, runtime_checkable' '''@runtime_checkable class MyProtocol(Protocol): def meth(self): pass ''' 50000 loops, best of 5: 9.28 usec per loop $ ./python.exe -m timeit -s 'from typing import Protocol, runtime_checkable' '''class MyProtocol(Protocol): def meth(self): pass ''' 50000 loops, best of 5: 9.05 usec per loop Patched: $ ./python.exe -m timeit -s 'from typing import Protocol, runtime_checkable' '''@runtime_checkable class MyProtocol(Protocol): def meth(self): pass ''' 50000 loops, best of 5: 7.69 usec per loop $ ./python.exe -m timeit -s 'from typing import Protocol, runtime_checkable' '''class MyProtocol(Protocol): def meth(self): pass ''' 50000 loops, best of 5: 7.78 usec per loop This was on a debug build though and I haven't compared it with versions where Protocol just accessed `.__annotations__` directly, and it's not a huge difference, so I don't think it's worth calling out the optimization too much. A downside of this change is that any errors that happen during the determination of attributes now happen only the first time isinstance() is called. This seems OK since these errors happen only in fairly exotic circumstances. Another downside is that any attributes added after class initialization now get picked up as protocol members. This came up in the typing test suite due to `@final`, but may cause issues elsewhere too.
|
If I understand correctly, this will significantly slow down the first Since it's a micro-benchmark anyway, we could possibly even adjust that benchmark so that it does an |
|
I think it might already do some warmup runs? That makes sense for the specializing interpreter anyway. I'll post on Discord. |
|
It's a shame we can't use |
|
Michael confirmed that it does a warmup run by default. |
|
When you're done making the requested changes, leave the comment: |
|
I have made the requested changes; please review again |
|
Thanks for making the requested changes! @AlexWaygood: please review the changes made to this pull request. |
|
I realize this also impacts the note in the docs (https://docs.python.org/3.14/library/typing.html#typing.runtime_checkable) "The members of a runtime-checkable protocol are now considered “frozen” at runtime as soon as the class has been created." This no longer quite holds; they are now frozen as of the first isinstance() check. Not a great compatibility story; I may want to think more about how to handle this. |
There was a problem hiding this comment.
This change does make sense to me. I agree that the "Protocol classes are now frozen at class-creation time" note in the docs no longer holds true, but I think the message to users from that note is "don't monkeypatch attributes onto Protocol classes". If users are following that advice, I don't think they would be broken by this change...?
We might be able to just edit the existing "Changed in Python 3.12" note to make it a bit more vague about when protocols are frozen, rather than adding a "Changed in Python 3.14" note? Not sure.
|
An alternative that keeps us from importing annotationlib too early is to do something like So we only import annotationlib in the case where we actually need deferred annotations. I think I prefer that for compatibility. |
|
Ah that sounds like an appealing alternative. Let's keep the new regression test you added as well, though! |
Misc/NEWS.d/next/Library/2025-04-16-06-50-13.gh-issue-132493.XvnL7t.rst
Outdated
Show resolved
Hide resolved
|
probably need to change the PR title ;) |
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
|
|
|
I don't think that's this PR |
gh-132494 made typing.py eagerly import annotationlib again because
typing contains several protocols. Avoid this by using annotationlib only if
.__annotations__fails.An earlier version of this PR instead made it so we compute the protocol attrs only when needed (generally on the first
isinstance()call). That would make startup faster, but has some compatibility implications in cases where attributes get added to the class later, so I think this version is safer.