Skip to content

test: Filter tests based on support.requires(...) and supported resources #412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

jcfr
Copy link
Contributor

@jcfr jcfr commented May 31, 2025

This update ensures that only tests requiring supported resources are executed. Specifically:

  • Introduces SUPPORTED_TEST_RESOURCES (defaulting to network) and uses it to drive both test filtering and the -u option passed to regrtest.py.

  • Updates UnitTests.cmake to parse unindented support.requires(...) calls in test scripts, extracting the first argument (i.e., the required resource).

  • Skips any test that requires a resource not listed in SUPPORTED_TEST_RESOURCES.

  • Adds support for hardcoded resource mappings (e.g., for tests like test_zipfile64 where the requires(...) call spans multiple lines).

  • Emits informative messages when tests are skipped due to unmet resource requirements.

Example of output for Python 3.11:

[...]

-- Supported test resources: network
--   Ignoring 'test_curses' requiring 'curses' resource
--   Ignoring 'test_ossaudiodev' requiring 'audio' resource
--   Ignoring 'test_tix' requiring 'gui' resource
--   Ignoring 'test_tk' requiring 'gui' resource
--   Ignoring 'test_ttk_guionly' requiring 'gui' resource
--   Ignoring 'test_winsound' requiring 'audio' resource
--   Ignoring 'test_zipfile64' requiring 'extralargefile' resource
-- Discovered 407 tests (skipping 7)

[...]

Working toward addressing:

…rces

This update ensures that only tests requiring supported resources are executed.
Specifically:

* Introduces `SUPPORTED_TEST_RESOURCES` (defaulting to `network`) and
  uses it to drive both test filtering and the `-u` option passed to `regrtest.py`.

* Updates `UnitTests.cmake` to parse unindented `support.requires(...)` calls
  in test scripts, extracting the first argument (i.e., the required resource).

* Skips any test that requires a resource not listed in `SUPPORTED_TEST_RESOURCES`.

* Adds support for hardcoded resource mappings (e.g., for tests like `test_zipfile64`
  where the `requires(...)` call spans multiple lines).

* Emits informative messages when tests are skipped due to unmet resource requirements.
@jcfr
Copy link
Contributor Author

jcfr commented May 31, 2025

@jamesobutler This follows up on this morning discussion ✨ and is a step toward requiring tests to pass before integrating a pull request 👌

@jcfr jcfr merged commit 6cb882e into python-cmake-buildsystem:master May 31, 2025
24 checks passed
@jcfr jcfr deleted the exclude-tests-using-unsupported-resources branch May 31, 2025 01:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

1 participant