Skip to content

regr_test.py: improve interaction between --all and --quiet flags #9349

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 10, 2022

Conversation

JelleZijlstra
Copy link
Member

cf. #9348 cc @jab

@JelleZijlstra
Copy link
Member Author

pyright 1.1.278
/home/runner/work/typeshed/typeshed/test_cases/stdlib/typing/check_regression_issue_9296.py
  /home/runner/work/typeshed/typeshed/test_cases/stdlib/typing/check_regression_issue_9296.py:16:12 - error: Cannot access member "regixster" for type "Type[MyKeysView[KT@MyKeysView]]"
    Member "regixster" is unknown (reportGeneralTypeIssues)
  /home/runner/work/typeshed/typeshed/test_cases/stdlib/typing/check_regression_issue_9296.py:16:1 - error: Type of "regixster" is unknown (reportUnknownMemberType)

It works as expected.

@AlexWaygood
Copy link
Member

AlexWaygood commented Dec 10, 2022

And we have a failing mypy run as well: https://github.com/python/typeshed/actions/runs/3665455929/jobs/6196543252

(Though it looks like I could probably sink a little more work into improving that traceback)

@JelleZijlstra
Copy link
Member Author

Is the reason the failure shows up ~16 times that we run mypy in a matrix (Python version, OS) and report the failure every time?

@AlexWaygood
Copy link
Member

Is the reason the failure shows up ~16 times that we run mypy in a matrix (Python version, OS) and report the failure every time?

Yup! Just pushed a commit to this PR branch that hopefully clarifies things a bit?

The issue was that the new --quiet flag didn't really interact very well with the --all flag

@JelleZijlstra
Copy link
Member Author

Nice, feel free to hijack this PR for your change, or just open a new PR.

A more radical change could be to show only the first failure if there are multiple identical ones. After all, in most cases test cases are going to fail in the same way on multiple runs.

@AlexWaygood
Copy link
Member

After all, in most cases test cases are going to fail in the same way on multiple runs.

This is true, but it won't always be the case. I feel like this solution is ~good enough for now. I'm gonna commence hijacking ;)

@AlexWaygood AlexWaygood marked this pull request as ready for review December 10, 2022 21:30
@AlexWaygood AlexWaygood changed the title [DO NOT MERGE] check that test case is working as expected regr_test.py: improve interaction between --all and --quiet flags Dec 10, 2022
@AlexWaygood AlexWaygood merged commit 388d783 into main Dec 10, 2022
@AlexWaygood AlexWaygood deleted the JelleZijlstra-patch-1 branch December 10, 2022 21:39
AlexWaygood added a commit that referenced this pull request Jan 30, 2023
It looks like in #9382, I reintroduced the bug that I previously fixed in #9349
AlexWaygood added a commit that referenced this pull request Jan 30, 2023
It looks like in #9382, I reintroduced the bug that I previously fixed in #9349
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants