Improve pyright verification of third-party test cases in CI #9650
Conversation
Co-authored-by: Avasam <samuel.06@hotmail.com>
594ebe9 to
fd43709
Compare
…pyright now fails CI
…ow that pyright now fails CI" This reverts commit 0891ef1.
|
Here's a failing run showing that, with these changes, pyright will once again error for unused |
| with: | ||
| version: ${{ steps.pyright_version.outputs.value }} | ||
| python-platform: ${{ matrix.python-platform }} | ||
| python-version: ${{ matrix.python-version }} | ||
| no-comments: ${{ matrix.python-version != '3.10' || matrix.python-platform != 'Linux' }} # Having each job create the same comment is too noisy. | ||
| project: ./pyrightconfig.stricter.json |
There was a problem hiding this comment.
I think running stricter first would fail faster in most cases. Since they both run on all files not excluded by the stricter settings. Stricter runs on less files and a failure with basic config would also result in a failure with stricter ones.
But this job is also pretty fast anyway.
There was a problem hiding this comment.
Also I've found it pretty annoying in the past when we're working to enable new pyright lints, the way you have to play "pyright whackamole" with our current setup. You think you've got all the pyright errors sorted, but then you find out, oh no, it was only the strict-config errors you've got sorted! Most of typeshed is still to go, it's just we never even got to the second step 😶
This order feels more intuitive to me :)
As seen in #9623, following some recent changes to our pyright config, pyright no longer verifies our checks against false-negative errors that we have in our third-party test cases. (Our standard-library test cases are fine.) Luckily, the latest release of pyright means that we can use the same pyright config for our third-party test cases as we do for our standard-library test cases, by using wildcards in
includeandexcludepatterns. This means that we can remove a no-longer-necessary test fromcheck_consistent.py, and it means that we no longer have to have# pyright: reportUnnecessaryTypeIgnoreComment=truecomments at the top of every third-party test-case file.Also in this PR:
check_consistent.py, ensuring that.pyfiles are only found in the@tests/test_cases/directory, and not at the top level of the@tests/directory. This isn't necessarily obvious, and caused confusion inprotobuf: Annotatewell_known_types.pyi#9323.Co-authored-by: Avasam samuel.06@hotmail.com