Skip to content

pytype_test.py: Use importlib.metadata instead of pkg_resources #10391

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/typecheck_typeshed_code.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: "3.9"
python-version: "3.10"
cache: pip
cache-dependency-path: requirements-tests.txt
- run: pip install -r requirements-tests.txt
Expand All @@ -66,5 +66,5 @@ jobs:
with:
version: ${{ steps.pyright_version.outputs.value }}
python-platform: ${{ matrix.python-platform }}
python-version: "3.9"
python-version: "3.10"
project: ./pyrightconfig.scripts_and_tests.json
1 change: 0 additions & 1 deletion requirements-tests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,3 @@ typing-extensions==4.6.3

# Type stubs used to type check our scripts.
types-pyyaml>=6.0.12.7
types-setuptools>=67.5.0.0
4 changes: 2 additions & 2 deletions tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ objects at runtime.
in the `tests` and `scripts` directories.

To run the tests, follow the [setup instructions](../CONTRIBUTING.md#preparing-the-environment)
in the `CONTRIBUTING.md` document. In particular, you have to run with Python 3.9+.
in the `CONTRIBUTING.md` document. In particular, you have to run with Python 3.10+.

In order for `pytype_test` and `pyright_test` to work correctly, some third-party stubs
may require extra dependencies external to typeshed to be installed in your virtual environment
Expand Down Expand Up @@ -72,7 +72,7 @@ for this script.

Note: this test cannot be run on Windows
systems unless you are using Windows Subsystem for Linux.
It also requires a Python version < 3.11 as pytype does not yet support
It can currently only be run on Python 3.10 as pytype does not yet support
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could avoid this if we added the importlib_metadata backport as a test dependency

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer the narrow requirements in this case. I would be surprised if pytype_test is much run manually and an extra requirement can always be a bit of a pain.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, though I don't have a strong opinion at all

Python 3.11 and above.

Run using:
Expand Down
25 changes: 15 additions & 10 deletions tests/pytype_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,22 @@
from __future__ import annotations

import argparse
import importlib.metadata
import os
import sys
import traceback
from collections import defaultdict
from collections.abc import Iterable, Sequence

import pkg_resources
from packaging.requirements import Requirement

from parse_metadata import read_dependencies

if sys.platform == "win32":
print("pytype does not support Windows.", file=sys.stderr)
sys.exit(1)
if sys.version_info >= (3, 11):
print("pytype does not support Python 3.11+ yet.", file=sys.stderr)
if sys.version_info[:2] != (3, 10):
print("pytype_test.py can currently only be run on Python 3.10.", file=sys.stderr)
sys.exit(1)

# pytype is not py.typed https://github.com/google/pytype/issues/1325
Expand Down Expand Up @@ -165,15 +167,18 @@ def get_missing_modules(files_to_test: Sequence[str]) -> Iterable[str]:
except ValueError:
continue
stub_distributions.add(parts[idx + 1])

dist_to_pkg_map = defaultdict(list)
for dist, pkg_list in importlib.metadata.packages_distributions().items():
for pkg in pkg_list:
dist_to_pkg_map[pkg].append(dist)

missing_modules = set()
for distribution in stub_distributions:
for pkg in read_dependencies(distribution).external_pkgs:
egg_info = pkg_resources.get_distribution(pkg).egg_info
assert isinstance(egg_info, str)
# See https://stackoverflow.com/a/54853084
top_level_file = os.path.join(egg_info, "top_level.txt")
with open(top_level_file) as f:
missing_modules.update(f.read().splitlines())
for external_req in read_dependencies(distribution).external_pkgs:
pkg = Requirement(external_req).name
missing_modules.update(dist_to_pkg_map[pkg])

test_dir = os.path.dirname(__file__)
exclude_list = os.path.join(test_dir, "pytype_exclude_list.txt")
with open(exclude_list) as f:
Expand Down
2 changes: 1 addition & 1 deletion tests/typecheck_typeshed.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
ReturnCode: TypeAlias = int

SUPPORTED_PLATFORMS = ("linux", "darwin", "win32")
SUPPORTED_VERSIONS = ("3.12", "3.11", "3.10", "3.9")
SUPPORTED_VERSIONS = ("3.12", "3.11", "3.10")
LOWEST_SUPPORTED_VERSION = min(SUPPORTED_VERSIONS, key=lambda x: int(x.split(".")[1]))
DIRECTORIES_TO_TEST = ("scripts", "tests")
EMPTY: list[str] = []
Expand Down