Skip to content

Check what made hang #5745

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from
Closed

Check what made hang #5745

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
4eb2136
revert a lot
youknowone Apr 25, 2025
71d8e94
DO NOT MERGE! reduce ci not to waste them
youknowone Apr 25, 2025
65ab3b5
check if other platforms also hang
youknowone Apr 27, 2025
d9c5c89
remove platform-independent testes
youknowone Apr 29, 2025
212944a
bisect try #1
youknowone Apr 29, 2025
2a8e0f1
try #2
youknowone Apr 29, 2025
8aa9d7d
try 3
youknowone Apr 29, 2025
f2f15c7
try 3
youknowone Apr 29, 2025
959ffed
try 4
youknowone Apr 29, 2025
812a9c1
try 5
youknowone Apr 29, 2025
2df2072
fully back
youknowone Apr 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .cspell.dict/rust-more.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,6 @@ unic
unistd
unraw
unsync
wasip1
wasip2
wasmbind
wasmtime
widestring
Expand Down
292 changes: 3 additions & 289 deletions .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ concurrency:
cancel-in-progress: true

env:
CARGO_ARGS: --no-default-features --features stdlib,importlib,stdio,encodings,sqlite,ssl
CARGO_ARGS: --no-default-features --features stdlib,importlib,encodings,sqlite,ssl
# Skip additional tests on Windows. They are checked on Linux and MacOS.
# test_glob: many failing tests
# test_io: many failing tests
Expand Down Expand Up @@ -112,130 +112,6 @@ env:
PYTHON_VERSION: "3.13.1"

jobs:
rust_tests:
if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip:ci') }}
env:
RUST_BACKTRACE: full
name: Run rust tests
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
fail-fast: false
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- uses: Swatinem/rust-cache@v2

- name: Set up the Windows environment
shell: bash
run: |
git config --system core.longpaths true
cargo install --target-dir=target -v cargo-vcpkg
cargo vcpkg -v build
if: runner.os == 'Windows'
- name: Set up the Mac environment
run: brew install autoconf automake libtool
if: runner.os == 'macOS'

- name: run clippy
run: cargo clippy ${{ env.CARGO_ARGS }} --workspace --all-targets --exclude rustpython_wasm -- -Dwarnings

- name: run rust tests
run: cargo test --workspace --exclude rustpython_wasm --verbose --features threading ${{ env.CARGO_ARGS }}
if: runner.os != 'macOS'
- name: run rust tests
run: cargo test --workspace --exclude rustpython_wasm --exclude rustpython-jit --verbose --features threading ${{ env.CARGO_ARGS }}
if: runner.os == 'macOS'

- name: check compilation without threading
run: cargo check ${{ env.CARGO_ARGS }}

- name: Test example projects
run:
cargo run --manifest-path example_projects/barebone/Cargo.toml
cargo run --manifest-path example_projects/frozen_stdlib/Cargo.toml
if: runner.os == 'Linux'

- name: prepare AppleSilicon build
uses: dtolnay/rust-toolchain@stable
with:
target: aarch64-apple-darwin
if: runner.os == 'macOS'
- name: Check compilation for Apple Silicon
run: cargo check --target aarch64-apple-darwin
if: runner.os == 'macOS'
- name: prepare iOS build
uses: dtolnay/rust-toolchain@stable
with:
target: aarch64-apple-ios
if: runner.os == 'macOS'
- name: Check compilation for iOS
run: cargo check --target aarch64-apple-ios
if: runner.os == 'macOS'

exotic_targets:
if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip:ci') }}
name: Ensure compilation on various targets
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
target: i686-unknown-linux-gnu

- name: Install gcc-multilib and musl-tools
run: sudo apt-get update && sudo apt-get install gcc-multilib musl-tools
- name: Check compilation for x86 32bit
run: cargo check --target i686-unknown-linux-gnu

- uses: dtolnay/rust-toolchain@stable
with:
target: aarch64-linux-android

- name: Check compilation for android
run: cargo check --target aarch64-linux-android

- uses: dtolnay/rust-toolchain@stable
with:
target: aarch64-unknown-linux-gnu

- name: Install gcc-aarch64-linux-gnu
run: sudo apt install gcc-aarch64-linux-gnu
- name: Check compilation for aarch64 linux gnu
run: cargo check --target aarch64-unknown-linux-gnu

- uses: dtolnay/rust-toolchain@stable
with:
target: i686-unknown-linux-musl

- name: Check compilation for musl
run: cargo check --target i686-unknown-linux-musl

- uses: dtolnay/rust-toolchain@stable
with:
target: x86_64-unknown-freebsd

- name: Check compilation for freebsd
run: cargo check --target x86_64-unknown-freebsd

- uses: dtolnay/rust-toolchain@stable
with:
target: x86_64-unknown-freebsd

- name: Check compilation for freeBSD
run: cargo check --target x86_64-unknown-freebsd

- name: Prepare repository for redox compilation
run: bash scripts/redox/uncomment-cargo.sh
- name: Check compilation for Redox
uses: coolreader18/redoxer-action@v1
with:
command: check
args: --ignore-rust-version

snippets_cpython:
if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip:ci') }}
env:
Expand All @@ -244,7 +120,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
os: [ubuntu-latest]
fail-fast: false
steps:
- uses: actions/checkout@v4
Expand Down Expand Up @@ -275,169 +151,7 @@ jobs:
- name: run snippets
run: python -m pip install -r requirements.txt && pytest -v
working-directory: ./extra_tests
- if: runner.os == 'Linux'
name: run cpython platform-independent tests
- name: run cpython platform-independent tests
run:
target/release/rustpython -m test -j 1 -u all --slowest --fail-env-changed -v ${{ env.PLATFORM_INDEPENDENT_TESTS }}
- if: runner.os == 'Linux'
name: run cpython platform-dependent tests (Linux)
run: target/release/rustpython -m test -j 1 -u all --slowest --fail-env-changed -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }}
- if: runner.os == 'macOS'
name: run cpython platform-dependent tests (MacOS)
run: target/release/rustpython -m test -j 1 --slowest --fail-env-changed -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }} ${{ env.MACOS_SKIPS }}
- if: runner.os == 'Windows'
name: run cpython platform-dependent tests (windows partial - fixme)
run:
target/release/rustpython -m test -j 1 --slowest --fail-env-changed -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }} ${{ env.WINDOWS_SKIPS }}
- if: runner.os != 'Windows'
name: check that --install-pip succeeds
run: |
mkdir site-packages
target/release/rustpython --install-pip ensurepip --user
target/release/rustpython -m pip install six
- if: runner.os != 'Windows'
name: Check that ensurepip succeeds.
run: |
target/release/rustpython -m ensurepip
target/release/rustpython -c "import pip"
- if: runner.os != 'Windows'
name: Check if pip inside venv is functional
run: |
target/release/rustpython -m venv testvenv
testvenv/bin/rustpython -m pip install wheel
- name: Check whats_left is not broken
run: python -I whats_left.py

lint:
name: Check Rust code with rustfmt and clippy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt, clippy
- name: run rustfmt
run: cargo fmt --check
- name: run clippy on wasm
run: cargo clippy --manifest-path=wasm/lib/Cargo.toml -- -Dwarnings
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: install ruff
run: python -m pip install ruff==0.0.291 # astral-sh/ruff#7778
- name: Ensure docs generate no warnings
run: cargo doc
- name: run python lint
run: ruff extra_tests wasm examples --exclude='./.*',./Lib,./vm/Lib,./benches/ --select=E9,F63,F7,F82 --show-source
- name: install prettier
run: yarn global add prettier && echo "$(yarn global bin)" >>$GITHUB_PATH
- name: check wasm code with prettier
# prettier doesn't handle ignore files very well: https://github.com/prettier/prettier/issues/8506
run: cd wasm && git ls-files -z | xargs -0 prettier --check -u
# Keep cspell check as the last step. This is optional test.
- name: install extra dictionaries
run: npm install @cspell/dict-en_us @cspell/dict-cpp @cspell/dict-python @cspell/dict-rust @cspell/dict-win32 @cspell/dict-shell
- name: spell checker
uses: streetsidesoftware/cspell-action@v6
with:
files: '**/*.rs'
incremental_files_only: true

miri:
if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip:ci') }}
name: Run tests under miri
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@master
with:
toolchain: nightly
components: miri

- uses: Swatinem/rust-cache@v2
- name: Run tests under miri
# miri-ignore-leaks because the type-object circular reference means that there will always be
# a memory leak, at least until we have proper cyclic gc
run: MIRIFLAGS='-Zmiri-ignore-leaks' cargo +nightly miri test -p rustpython-vm -- miri_test

wasm:
if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip:ci') }}
name: Check the WASM package and demo
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable

- uses: Swatinem/rust-cache@v2
- name: install wasm-pack
run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
- name: install geckodriver
run: |
wget https://github.com/mozilla/geckodriver/releases/download/v0.36.0/geckodriver-v0.36.0-linux64.tar.gz
mkdir geckodriver
tar -xzf geckodriver-v0.36.0-linux64.tar.gz -C geckodriver
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- run: python -m pip install -r requirements.txt
working-directory: ./wasm/tests
- uses: actions/setup-node@v4
with:
cache: "npm"
cache-dependency-path: "wasm/demo/package-lock.json"
- name: run test
run: |
export PATH=$PATH:`pwd`/../../geckodriver
npm install
npm run test
env:
NODE_OPTIONS: "--openssl-legacy-provider"
working-directory: ./wasm/demo
- uses: mwilliamson/setup-wabt-action@v3
with: { wabt-version: "1.0.36" }
- name: check wasm32-unknown without js
run: |
cd wasm/wasm-unknown-test
cargo build --release --verbose
if wasm-objdump -xj Import target/wasm32-unknown-unknown/release/wasm_unknown_test.wasm; then
echo "ERROR: wasm32-unknown module expects imports from the host environment" >2
fi
- name: build notebook demo
if: github.ref == 'refs/heads/release'
run: |
npm install
npm run dist
mv dist ../demo/dist/notebook
env:
NODE_OPTIONS: "--openssl-legacy-provider"
working-directory: ./wasm/notebook
- name: Deploy demo to Github Pages
if: success() && github.ref == 'refs/heads/release'
uses: peaceiris/actions-gh-pages@v4
env:
ACTIONS_DEPLOY_KEY: ${{ secrets.ACTIONS_DEMO_DEPLOY_KEY }}
PUBLISH_DIR: ./wasm/demo/dist
EXTERNAL_REPOSITORY: RustPython/demo
PUBLISH_BRANCH: master

wasm-wasi:
if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip:ci') }}
name: Run snippets and cpython tests on wasm-wasi
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
target: wasm32-wasip1

- uses: Swatinem/rust-cache@v2
- name: Setup Wasmer
uses: wasmerio/setup-wasmer@v3
- name: Install clang
run: sudo apt-get update && sudo apt-get install clang -y
- name: build rustpython
run: cargo build --release --target wasm32-wasip1 --features freeze-stdlib,stdlib --verbose
- name: run snippets
run: wasmer run --dir `pwd` target/wasm32-wasip1/release/rustpython.wasm -- `pwd`/extra_tests/snippets/stdlib_random.py
- name: run cpython unittest
run: wasmer run --dir `pwd` target/wasm32-wasip1/release/rustpython.wasm -- `pwd`/Lib/test/test_int.py
Loading
Loading