Skip to content

CI Migrate Linux ARM job to CirrusCI #25366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 13 additions & 23 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,27 +37,6 @@ jobs:
root: doc/_build/html
paths: .

linux-arm64:
machine:
image: ubuntu-2004:202101-01
resource_class: arm.medium
environment:
- OMP_NUM_THREADS: 2
- OPENBLAS_NUM_THREADS: 2
- CONDA_ENV_NAME: testenv
- LOCK_FILE: build_tools/circle/py39_conda_forge_linux-aarch64_conda.lock
steps:
- checkout
- run: ./build_tools/circle/checkout_merge_commit.sh
- restore_cache:
key: linux-arm64-{{ .Branch }}
- run: ./build_tools/circle/build_test_arm.sh
- save_cache:
key: linux-arm64-{{ .Branch }}
paths:
- ~/.cache/ccache
- ~/.cache/pip
- ~/scikit_learn_data
deploy:
docker:
- image: cimg/python:3.8.12
Expand All @@ -75,6 +54,17 @@ jobs:
bash build_tools/circle/push_doc.sh doc/_build/html/stable
fi

# This noop job is required for the pipeline to exist, so that the
# documentation related jobs can be triggered.
noop:
docker:
- image: cimg/python:3.8.12
steps:
- run: |
echo "This is no-op job for the pipeline to exist, so that it triggers "
echo "Circle CI jobs pushing the artifacts of the documentation built "
echo "via GitHub actions."

workflows:
version: 2

Expand All @@ -90,10 +80,10 @@ workflows:
requires:
- doc

linux-arm64:
noop:
when:
equal: [ "none", << pipeline.parameters.GITHUB_RUN_URL >> ]
# Prevent double execution of this job: on push
# by default and when triggered by the workflow
jobs:
- linux-arm64
- noop
8 changes: 6 additions & 2 deletions .cirrus.star
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ def main(ctx):
return []

arm_wheel_yaml = "build_tools/cirrus/arm_wheel.yml"
arm_tests_yaml = "build_tools/cirrus/arm_tests.yml"

# Nightly jobs always run
if env.get("CIRRUS_CRON", "") == "nightly":
Expand All @@ -25,7 +26,10 @@ def main(ctx):
response = http.get(url).json()
commit_msg = response["message"]

if "[skip ci]" in commit_msg or ("[cd build]" not in commit_msg and "[cd build cirrus]" not in commit_msg):
if "[skip ci]" in commit_msg:
return []

return fs.read(arm_wheel_yaml)
if "[cd build]" in commit_msg or "[cd build cirrus]" in commit_msg:
return fs.read(arm_wheel_yaml)

return fs.read(arm_tests_yaml)
21 changes: 21 additions & 0 deletions build_tools/cirrus/arm_tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
linux_aarch64_test_task:
compute_engine_instance:
image_project: cirrus-images
image: family/docker-builder-arm64
architecture: arm64
platform: linux
cpu: 4
memory: 6G
env:
CONDA_ENV_NAME: testenv
OMP_NUM_THREADS: 2
OPENBLAS_NUM_THREADS: 2
LOCK_FILE: build_tools/cirrus/py39_conda_forge_linux-aarch64_conda.lock
CONDA_PKGS_DIRS: /root/.conda/pkgs
ccache_cache:
folder: /root/.cache/ccache
conda_cache:
folder: /root/.conda/pkgs
fingerprint_script: cat build_tools/cirrus/py39_conda_forge_linux-aarch64_conda.lock

test_script: bash build_tools/cirrus/build_test_arm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ https://conda.anaconda.org/conda-forge/noarch/pytest-forked-1.4.0-pyhd8ed1ab_1.t
https://conda.anaconda.org/conda-forge/linux-aarch64/matplotlib-3.6.2-py39ha65689a_0.tar.bz2#b4d712f422b5dad5259f38151be6f492
https://conda.anaconda.org/conda-forge/noarch/pyopenssl-23.0.0-pyhd8ed1ab_0.conda#d41957700e83bbb925928764cb7f8878
https://conda.anaconda.org/conda-forge/noarch/pytest-xdist-2.5.0-pyhd8ed1ab_0.tar.bz2#1fdd1f3baccf0deb647385c677a1a48e
https://conda.anaconda.org/conda-forge/noarch/urllib3-1.26.13-pyhd8ed1ab_0.conda#3078ef2359efd6ecadbc7e085c5e0592
https://conda.anaconda.org/conda-forge/noarch/urllib3-1.26.14-pyhd8ed1ab_0.conda#01f33ad2e0aaf6b5ba4add50dad5ad29
https://conda.anaconda.org/conda-forge/noarch/requests-2.28.1-pyhd8ed1ab_1.tar.bz2#089382ee0e2dc2eae33a04cc3c2bddb0
https://conda.anaconda.org/conda-forge/noarch/pooch-1.6.0-pyhd8ed1ab_0.tar.bz2#6429e1d1091c51f626b5dcfdd38bf429
https://conda.anaconda.org/conda-forge/linux-aarch64/scipy-1.10.0-py39hafab3e7_0.conda#65d57c881ae668b5372bb6ad5e08bb78
2 changes: 1 addition & 1 deletion build_tools/update_environments_and_lock_files.py
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ def remove_from(alist, to_remove):
},
{
"build_name": "py39_conda_forge",
"folder": "build_tools/circle",
"folder": "build_tools/cirrus",
"platform": "linux-aarch64",
"channel": "conda-forge",
"conda_dependencies": remove_from(
Expand Down