CI Adds CPython 3.10 testing [python-dev] #20942

thomasjpfan · 2021-09-04T20:29:31Z

This PR adds python 3.10 testing to GitHub actions. Currently numpy has a 3.10 wheel to use and scipy needs to be build from source.

With yearly Python releases, I think it is good to have this type of testing set up.

ogrisel · 2021-09-06T13:48:44Z

Thanks. The pip build error is not very helpful:

WARNING: Discarding file:///home/runner/work/scikit-learn/scikit-learn. Command errored out with exit status 1: /opt/hostedtoolcache/Python/3.10.0-rc.1/x64/bin/python /opt/hostedtoolcache/Python/3.10.0-rc.1/x64/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpo9d1isjo Check the logs for full command output.
ERROR: Command errored out with exit status 1: /opt/hostedtoolcache/Python/3.10.0-rc.1/x64/bin/python /opt/hostedtoolcache/Python/3.10.0-rc.1/x64/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpo9d1isjo Check the logs for full command output.

shall we run pip with --verbose when building scikit-learn (and scipy?).

ogrisel · 2021-09-06T13:49:18Z

This build would probably benefit from using ccache with a reusable ccache folder.

Edit: apparently ccache integration can be achieved with https://github.com/hendrikmuhs/ccache-action

ogrisel

It seems to be fast enough. @thomasjpfan why is this PR still in draft mode? Do you have other things in mind?

For scikit-learn, we could use cibuildwheel instead of a traditional build + test steps and publish the resulting wheel to https://anaconda.org/scipy-wheels-nightly/scikit-learn/files in addition to the other nightly builds.

But +1 to merge even without this, as long at it stays green :)

ogrisel · 2021-09-07T14:10:15Z

.github/workflows/cpython-nightly.yml

+          pip install --no-build-isolation -e .
+      - name: Test scikit-learn
+        run: |
+          cd /tmp && python -m pytest -n 2 --pyargs sklearn


Even if our OPENMP-based Cython code should now avoid oversubscription problems in docker containers, numpy/scipy might still use too many threads and slow down the execution.

To be safe I think we should set OMP_NUM_THREADS=2 / OPENBLAS_NUM_THREADS=2 globally for this workflow.

@thomasjpfan Shall we set the environment variable in this file as well?

The environment variables are set globally in lines 42-45:

env: INSTALL_NUMPY: ${{ matrix.install-numpy }} INSTALL_SCIPY: ${{ matrix.install-scipy }} OMP_NUM_THREADS: '2' OPENBLAS_NUM_THREADS: '2'

Would this work and be useful?

Suggested change

cd /tmp && python -m pytest -n 2 --pyargs sklearn

cd /tmp && python -m pytest -n $OMP_NUM_THREADS --pyargs sklearn

ogrisel · 2021-09-07T14:12:21Z

.github/workflows/cpython-nightly.yml

+        run: |
+          echo "::set-output name=commit_msg::$(git log --no-merges -1 --oneline)"
+
+  nightly:


Maybe rename to nightly-cpython or upstream-nighly.

ogrisel · 2021-09-07T14:15:24Z

.github/workflows/cpython-nightly.yml

+      - name: Install scikit-learn
+        run: |
+          export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH"
+          pip install --no-build-isolation -e .


To get more informative outputs in case of problems:

Suggested change

pip install --no-build-isolation -e .

pip install -v --no-build-isolation -e .

Also, it might be interesting to set SKLEARN_BUILD_PARALLEL=3 to speed up this step a bit.

thomasjpfan · 2021-09-10T13:22:38Z

It seems to be fast enough. @thomasjpfan why is this PR still in draft mode? Do you have other things in mind?

I forgot to change it while testing ccache. It's ready.

For scikit-learn, we could use cibuildwheel instead of a traditional build + test steps and publish the resulting wheel to https://anaconda.org/scipy-wheels-nightly/scikit-learn/files in addition to the other nightly builds.

I was going to wait for MacPython/scipy-wheels#124 to get merged before having our own wheels. Should we upload scikit-learn wheels for 3.10 without SciPy wheels?

ogrisel · 2021-09-15T14:04:58Z

I was going to wait for MacPython/scipy-wheels#124 to get merged before having our own wheels. Should we upload scikit-learn wheels for 3.10 without SciPy wheels?

It's true that scikit-learn's Python 3.10 nightly wheels won't be that useful without scipy's but we don't strictly need to wait for them to do it ourselves :)

ogrisel · 2021-09-15T14:07:02Z

But's let's merge this PR as it is as it's already useful the way it is and we can do the wheel uploading later.

thomasjpfan · 2021-10-01T21:14:51Z

Wheels update

There looks to be movement in getting scipy wheels built for 3.10: MAINT: add Python 3.10 wheels MacPython/scipy-wheels#127
Also numpy is starting to look into wheels for osx and windows: Check if Python 3.10.0 available for Mac and Windows. MacPython/numpy-wheels#134
Also no 32bit Python 3.10 wheels: MAINT: add Python 3.10 wheels MacPython/scipy-wheels#127 (comment)

jjerphan

Thanks for setting this workflow, @thomasjpfan.

I just have one naive suggestion.

jjerphan · 2021-10-08T14:27:03Z

.github/workflows/cpython-nightly.yml

+          pip install --no-build-isolation -e .
+      - name: Test scikit-learn
+        run: |
+          cd /tmp && python -m pytest -n 2 --pyargs sklearn


Would this work and be useful?

Suggested change

cd /tmp && python -m pytest -n 2 --pyargs sklearn

cd /tmp && python -m pytest -n $OMP_NUM_THREADS --pyargs sklearn

thomasjpfan · 2021-10-08T20:00:41Z

We may not need this anymore if we have #21232

Edit: I think it would be good to keep this around to always test on the latest dev build of Python. I am thinking of keeping this CI on Python 3.10 and then move to 3.11 when the Python RC comes up. The tests would be slow in the beginning because wheels are not built yet for NumPy, SciPy, or pandas during the early in the RC phase.

glemaitre · 2021-10-14T09:05:19Z

The tests would be slow in the beginning because wheels are not built yet for NumPy, SciPy, or pandas during the early in the RC phase.

We could make it a CRON job at night like the build that check all nightly builds?

ogrisel · 2021-10-26T09:47:30Z

I think it's good to test against the nightly versions of CPython / numpy / scipy irrespective of their specific version numbers.

thomasjpfan · 2022-03-10T03:54:43Z

Closing as we are testing on python 3.10 on azure now.

thomasjpfan added 4 commits September 4, 2021 16:10

MAINT Adds CPython 3.10 testing [python-dev]

8170540

MAINT Use 'wheel' instead for a flag

3c153db

MAINT Removes pybind11 when it is not needed [python-dev]

5f9152e

CLN Depend on pep517 more [python-dev]

c4b9a74

thomasjpfan marked this pull request as draft September 4, 2021 22:23

thomasjpfan added 8 commits September 4, 2021 21:08

CLN Check skipping

bed166b

WIP [python-dev]

51a4a32

WIP [python-dev]

ae99c72

WIP [python-dev]

84403c5

WIP [python-dev]

8c1992b

WIP [python-dev]

8b4095f

WIP [python-dev]

e7444ac

WIP [python-dev]

c98f8f2

thomasjpfan added 3 commits September 6, 2021 15:07

BUG Install wheel [python-dev]

d5df2ea

MAINT Adds cache

72bf47b

BLD [python-dev]

7d4d06f

ogrisel approved these changes Sep 7, 2021

View reviewed changes

CLN Address comments

a645d22

thomasjpfan marked this pull request as ready for review September 10, 2021 13:12

thomasjpfan changed the title ~~MAINT Adds CPython 3.10 testing [python-dev]~~ CI Adds CPython 3.10 testing [python-dev] Sep 10, 2021

github-actions bot added the Build / CI label Sep 10, 2021

thomasjpfan added 2 commits September 15, 2021 09:45

Merge remote-tracking branch 'upstream/main' into python-3.10-testing

189ddfb

ENH Actually make it nightly

8f2fb82

cmarmo added the Waiting for Reviewer label Oct 5, 2021

jjerphan approved these changes Oct 8, 2021

View reviewed changes

thomasjpfan marked this pull request as draft October 21, 2021 14:30

thomasjpfan closed this Mar 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI Adds CPython 3.10 testing [python-dev] #20942

CI Adds CPython 3.10 testing [python-dev] #20942

thomasjpfan commented Sep 4, 2021

ogrisel commented Sep 6, 2021

ogrisel commented Sep 6, 2021 •

edited

Loading

ogrisel left a comment •

edited

Loading

ogrisel Sep 7, 2021 •

edited

Loading

glemaitre Sep 15, 2021

thomasjpfan Sep 15, 2021

jjerphan Oct 8, 2021

ogrisel Sep 7, 2021

ogrisel Sep 7, 2021

thomasjpfan commented Sep 10, 2021 •

edited

Loading

ogrisel commented Sep 15, 2021

ogrisel commented Sep 15, 2021

thomasjpfan commented Oct 1, 2021 •

edited

Loading

jjerphan left a comment

jjerphan Oct 8, 2021

thomasjpfan commented Oct 8, 2021 •

edited

Loading

glemaitre commented Oct 14, 2021

ogrisel commented Oct 26, 2021

thomasjpfan commented Mar 10, 2022

	cd /tmp && python -m pytest -n 2 --pyargs sklearn
	cd /tmp && python -m pytest -n $OMP_NUM_THREADS --pyargs sklearn

	pip install --no-build-isolation -e .
	pip install -v --no-build-isolation -e .

CI Adds CPython 3.10 testing [python-dev] #20942

CI Adds CPython 3.10 testing [python-dev] #20942

Conversation

thomasjpfan commented Sep 4, 2021

ogrisel commented Sep 6, 2021

ogrisel commented Sep 6, 2021 • edited Loading

ogrisel left a comment • edited Loading

Choose a reason for hiding this comment

ogrisel Sep 7, 2021 • edited Loading

Choose a reason for hiding this comment

glemaitre Sep 15, 2021

Choose a reason for hiding this comment

thomasjpfan Sep 15, 2021

Choose a reason for hiding this comment

jjerphan Oct 8, 2021

Choose a reason for hiding this comment

ogrisel Sep 7, 2021

Choose a reason for hiding this comment

ogrisel Sep 7, 2021

Choose a reason for hiding this comment

thomasjpfan commented Sep 10, 2021 • edited Loading

ogrisel commented Sep 15, 2021

ogrisel commented Sep 15, 2021

thomasjpfan commented Oct 1, 2021 • edited Loading

Wheels update

jjerphan left a comment

Choose a reason for hiding this comment

jjerphan Oct 8, 2021

Choose a reason for hiding this comment

thomasjpfan commented Oct 8, 2021 • edited Loading

glemaitre commented Oct 14, 2021

ogrisel commented Oct 26, 2021

thomasjpfan commented Mar 10, 2022

ogrisel commented Sep 6, 2021 •

edited

Loading

ogrisel left a comment •

edited

Loading

ogrisel Sep 7, 2021 •

edited

Loading

thomasjpfan commented Sep 10, 2021 •

edited

Loading

thomasjpfan commented Oct 1, 2021 •

edited

Loading

thomasjpfan commented Oct 8, 2021 •

edited

Loading