Skip to content

Update test suite #2181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Sep 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
6650809
WiP pyscript.web tests pass with upytest.
ntoll Sep 10, 2024
ec20298
Test refactoring. WiP
ntoll Sep 13, 2024
762ae19
Completed refactor of old integration tests to new Python tests.
ntoll Sep 17, 2024
b35fb8f
Added comprehensive test suite for Python based pyodide module.
ntoll Sep 19, 2024
806e574
Black.
ntoll Sep 19, 2024
177cbb9
Rebuld and check websocket attribute assignment via init and as attri…
ntoll Sep 23, 2024
091da6e
Update when tests to allow for worker round trips.
ntoll Sep 23, 2024
4a6c5fa
Post build
ntoll Sep 23, 2024
790fd26
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 23, 2024
ae66d13
Fixed config issue via polyscript update (#2182)
WebReflection Sep 24, 2024
e3c572d
WiP pyscript.web tests pass with upytest.
ntoll Sep 10, 2024
aa83e6c
Test refactoring. WiP
ntoll Sep 13, 2024
263f1fc
Completed refactor of old integration tests to new Python tests.
ntoll Sep 17, 2024
0eb8c43
Added comprehensive test suite for Python based pyodide module.
ntoll Sep 19, 2024
6c81b5f
Black.
ntoll Sep 19, 2024
43f25fd
Rebuld and check websocket attribute assignment via init and as attri…
ntoll Sep 23, 2024
344e54a
Update when tests to allow for worker round trips.
ntoll Sep 23, 2024
f24a2cf
Post build
ntoll Sep 23, 2024
6e52918
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 23, 2024
06216ac
Merge branch 'update-test-suite' of github.com:pyscript/pyscript into…
ntoll Sep 24, 2024
abc3a3b
Fix a couple of timing issues in display and web tests.
ntoll Sep 24, 2024
0d7ef9d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 24, 2024
57e023e
Black
ntoll Sep 24, 2024
58d905d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 24, 2024
7da2ba6
Add integration tests to Makefile (and CI)
ntoll Sep 24, 2024
f826d0b
Remove un-needed upload action.
ntoll Sep 24, 2024
af6cc2b
Ensure fails are properly logged as an array. Remove the explicit tes…
ntoll Sep 24, 2024
aa70295
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 24, 2024
19bc612
Directory reorg/refactor. Updated docs.
ntoll Sep 25, 2024
60622c9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 25, 2024
c52a695
Bump polyscript.
ntoll Sep 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 1 addition & 13 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,12 +69,7 @@ jobs:
make setup

- name: Build
run: make build

- name: Integration Tests
#run: make test-integration-parallel
run: |
make test-integration
run: make build # Integration tests run in the build step.

- uses: actions/upload-artifact@v4
with:
Expand All @@ -83,10 +78,3 @@ jobs:
pyscript.core/dist/
if-no-files-found: error
retention-days: 7

- uses: actions/upload-artifact@v4
if: success() || failure()
with:
name: test_results
path: test_results/
if-no-files-found: error
14 changes: 4 additions & 10 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ all:
@echo "make clean - clean up auto-generated assets."
@echo "make build - build PyScript."
@echo "make precommit-check - run the precommit checks (run eslint)."
@echo "make test-integration - run all integration tests sequentially."
@echo "make test - run all automated tests in playwright."
@echo "make fmt - format the code."
@echo "make fmt-check - check the code formatting.\n"

Expand Down Expand Up @@ -62,15 +62,9 @@ build:
precommit-check:
pre-commit run --all-files

# Run all integration tests sequentially.
test-integration:
mkdir -p test_results
pytest -vv $(ARGS) pyscript.core/tests/integration/ --log-cli-level=warning --junitxml=test_results/integration.xml

# Run all integration tests in parallel.
test-integration-parallel:
mkdir -p test_results
pytest --numprocesses auto -vv $(ARGS) pyscript.core/tests/integration/ --log-cli-level=warning --junitxml=test_results/integration.xml
# Run all automated tests in playwright.
test:
cd pyscript.core && npm run test:integration

# Format the code.
fmt: fmt-py
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ Read the [contributing guide](https://docs.pyscript.net/latest/contributing/) to

Check out the [developing process](https://docs.pyscript.net/latest/developers/) documentation for more information on how to setup your development environment.

For technical details of the code, please see the [README](pyscript.core/README) in `pyscript.core`.

## Governance

The [PyScript organization governance](https://github.com/pyscript/governance) is documented in a separate repository.
Expand Down
16 changes: 14 additions & 2 deletions pyscript.core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,25 @@ make setup

This will create a tests environment [in the root of the project, named `./env`]and install all the dependencies needed to run the tests.

After the command has completed and the tests environment has been created, you can run the **integration tests** with
After the command has completed and the tests environment has been created, you can run the **automated tests** with
the following command:

```
make test-integration
make test
```

(This essentially runs the `npm run test:integration` command in the right place. This is defined in PyScript's `package.json` file.)

Tests are found in the `tests` directory. These are organised into three locations:

1. `python` - the Python based test suite to exercise Python code **within** PyScript.
2. `javascript` - JavaScript tests to exercise PyScript itself, in the browser.
3. `manual` - containing tests to run manually in a browser, due to the complex nature of the tests.

We use [Playwright](https://playwright.dev/) to automate the running of the Python and JavaScript test suites. We use [uPyTest](https://github.com/ntoll/upytest) as a test framework for the Python test suite. uPyTest is a "PyTest inspired" framework for running tests in the browser on both MicroPython and Pyodide.

The automated (Playwright) tests are specified in the `tests/integration.spec.js` file.

## `pyscript` python package

The `pyscript` package available in _Python_ lives in the folder `src/stdlib/pyscript/`.
Expand Down
Loading