conformance tests: drop timing information #2070
Conversation
Timing information is nice to have but the current data is not very reliable, as it is the timing on whatever machine was used by the person who most recently regenerated the results. It's also a big source of merge conflicts, basically meaning that every pair of changes to the conformance suite will merge conflict with each other. To make it more pleasant to work with the conformance suite, let's just drop the timings.
|
I agree that it's a big source of issues with merge conflicts. I have always liked it being there as a user (even before Zuban). I think the only other option is to add a flag like |
|
There is an existing flag, I think it was |
|
I'm not against removing the timing information. I just always liked that information as a user, even if it was a bit imprecise. For example I would have guessed pyre to be faster than Mypy/Pyright, but that does not seem true at all (at least in in this case). I don't think the timings need to be that precise, but feel free to merge anyway. |
Timing information is nice to have but the current data is not very reliable,
as it is the timing on whatever machine was used by the person who most recently
regenerated the results. It's also a big source of merge conflicts, basically
meaning that every pair of changes to the conformance suite will merge conflict
with each other.
To make it more pleasant to work with the conformance suite, let's just drop the timings.