Add a benchmark for runtime-checkable protocols #280
Conversation
I did a fair amount of searching through grep.app while writing this benchmark to see what a typical runtime-checkable protocol is in the wild. The majority of runtime-checkable protocols in the wild appear to have 5 members or fewer. However, there are a few codebases that make heavy use of runtime-checkable protocols, and they often have "monster protocols" like the one you describe, which may have truly awful performance (and will probably have worse performance in 3.12 than they did in 3.11). One example of a codebase making heavy use of runtime-checkable protocols: https://github.com/feeluown/FeelUOwn/blob/master/feeluown/library/model_protocol.py. So, I'm not sure what the best thing to do here is! |
pyperformance/data-files/benchmarks/bm_typing_runtime_protocols/run_benchmark.py
Outdated
Show resolved
Hide resolved
pyperformance/data-files/benchmarks/bm_typing_runtime_protocols/run_benchmark.py
Outdated
Show resolved
Hide resolved
pyperformance/data-files/benchmarks/bm_typing_runtime_protocols/run_benchmark.py
Outdated
Show resolved
Hide resolved
Thanks for the review! I think I addressed all the comments. |
We had a longstanding issue open at CPython complaining that the performance of
isinstance()checks against runtime-checkable protocols was a serious bottleneck for their programme. The performance of theseisinstance()checks has been significantly improved for 3.12 compared to 3.11, but there's currently no code (to my knowledge) in pyperformance that significantly exercisesisinstance()checks against runtime-checkable protocols.Work towards #105.