MAINT unpin cython in CI config #27627

ogrisel · 2023-10-20T08:37:38Z

#27086 was closed as Cython 3.0.3 fixed all the Cython 3 related performance regressions we observed in the benchmarks.

So let's unpin Cython.

@lesteve I also updated the script to make it easy to display the commands and their outputs.

Note: the conda-lock command for "doc" runs indefinitely on my host, so I had to interrupt it with ctrl-C. There is no error message. I don't know how to debug this.

@lesteve or someone else could you please try to see if you can reproduce?

ogrisel · 2023-10-20T08:38:40Z

build_tools/azure/py38_conda_defaults_openblas_environment.yml

@@ -8,7 +8,7 @@ dependencies:
  - numpy=1.17.3  # min
  - blas[build=openblas]
  - scipy=1.5.0  # min
-  - cython<3.0.0
+  - cython=0.29.33  # min


Since other dependencies used min dependencies on this build I decided to follow this rule for cython as well.

github-actions · 2023-10-20T08:39:24Z

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 6ba70c2. Link to the linter CI: here}

lesteve · 2023-10-20T11:39:06Z

Note: the conda-lock command for "doc" runs indefinitely on my host, so I had to interrupt it with ctrl-C. There is no error message. I don't know how to debug this.

The doc lock file generation works fine on my machine and takes around 30 seconds.

ogrisel · 2023-10-20T12:06:39Z

@lesteve can you please push the results to this PR?

ogrisel · 2023-10-20T12:07:08Z

Once the CI has completed.

lesteve · 2023-10-20T14:49:56Z

I am unable to reproduce the weird scipy-dev error locally. On the other hand I get plenty of errors locally because of deprecated things in Python 3.12 some of them are not inside scikit-learn e.g. joblib, dateutil (dateutil/dateutil#1284). Probably the easiest thing to do would be to stay on Python 3.11 for scipy-dev for now (there is a chance that it fixes the scipy-dev CI if I have to guess).

scipy-dev error

============================= test session starts ==============================
platform linux -- Python 3.12.0, pytest-7.4.2, pluggy-1.3.0
rootdir: /home/vsts/work/tmp_folder
configfile: setup.cfg
plugins: forked-1.6.0, xdist-2.5.0, cov-4.1.0
gw0 I / gw1 I
gw0 [24972] / gw1 [24972]

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/_pytest/main.py", line 271, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/_pytest/main.py", line 325, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/xdist/dsession.py", line 117, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/xdist/dsession.py", line 140, in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/xdist/dsession.py", line 262, in worker_collectionfinish
INTERNALERROR>     self.sched.schedule()
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/xdist/scheduler/load.py", line 257, in schedule
INTERNALERROR>     self._send_tests(next(nodes), 1)
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/xdist/scheduler/load.py", line 269, in _send_tests
INTERNALERROR>     node.send_runtest_some(tests_per_node)
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/xdist/workermanage.py", line 284, in send_runtest_some
INTERNALERROR>     self.sendcommand("runtests", indices=indices)
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/xdist/workermanage.py", line 300, in sendcommand
INTERNALERROR>     self.channel.send((name, kwargs))
INTERNALERROR>   File "/usr/share/miniconda/envs/testvenv/lib/python3.12/site-packages/execnet/gateway_base.py", line 810, in send
INTERNALERROR>     raise OSError(f"cannot send to {self!r}")
INTERNALERROR> OSError: cannot send to <Channel id=1 closed>

ogrisel · 2023-10-20T14:54:31Z

Let me push this.

ogrisel · 2023-10-20T15:04:32Z

I pushed too early without waiting for the end of the PyPy tests of the previous build but they are still green after 1h55 of running, which is a net improvement compared to the last scheduled job that was crashing from the start:

https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=60198&view=logs&j=dfe99b15-50db-5d7b-b1e9-4105c42527cf&t=ef785ae2-496b-5b02-9f0e-07a6c3ab3081

Still they are too slow to run to wait for the review of this PR. Let's see the results of the next nightly PyPy build after the merge of this PR.

ogrisel · 2023-10-20T15:43:23Z

Yeah! the scipy_dev build is green with Python 3.11.

lesteve · 2023-10-20T18:45:05Z

doc and cirrus arm were passing on 68ea16b

I pushed a logging tweak (which I noticed also in #27622).

jjerphan

LGTM! Thank you for the recent work on investigating performance regression with Cython 3!

I think we can create "good first Cython issue" to replace interface inclusion which has been introduced in Cython 3 such as this one:

scikit-learn/sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors.pyx.tp

Lines 22 to 26 in fb6b9f5

    
           # TODO: change for `libcpp.algorithm.move` once Cython 3 is used 
        
           # Introduction in Cython: 
        
           # https://github.com/cython/cython/blob/05059e2a9b89bf6738a7750b905057e5b1e3fe2e/Cython/Includes/libcpp/algorithm.pxd#L47 #noqa 
        
           cdef extern from "<algorithm>" namespace "std" nogil: 
        
               OutputIt move[InputIt, OutputIt](InputIt first, InputIt last, OutputIt d_first) except + #noqa

What do you think?

Co-authored-by: Loïc Estève <loic.esteve@ymail.com>

ogrisel added 2 commits October 19, 2023 10:35

MAINT unpin Cython in lock files

558cd6d

Update lock files to unpin Cython 2

749db0e

ogrisel commented Oct 20, 2023

View reviewed changes

ogrisel changed the title ~~Ci unpin cython~~ MAINT unpin cython in CI config Oct 20, 2023

ogrisel added Build / CI cython labels Oct 20, 2023

ogrisel added 2 commits October 20, 2023 12:02

Merge remote-tracking branch 'origin/main' into ci-unpin-cython

7de7e75

Trigger CI for [scipy-dev] and [pypy]

5a073f7

ogrisel added 2 commits October 20, 2023 17:00

Temporary pin Python 3.11 for scipy_dev

b191620

Trigger CI for [scipy-dev]

eeaaf39

lesteve mentioned this pull request Oct 20, 2023

MNT Fix Python 3.12 deprecation warning joblib/joblib#1518

Merged

lesteve added 2 commits October 20, 2023 18:25

Update lock files [doc build] [cirrus arm]

68ea16b

Put logging when writing lock files rather than requirements

6ba70c2

lesteve approved these changes Oct 20, 2023

View reviewed changes

jjerphan approved these changes Oct 20, 2023

View reviewed changes

lesteve merged commit b22c706 into scikit-learn:main Oct 21, 2023

jjerphan deleted the ci-unpin-cython branch October 21, 2023 10:40

lesteve mentioned this pull request Oct 23, 2023

⚠️ CI failed on Linux_Nightly_PyPy.pypy3 ⚠️ #27416

Closed

glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Oct 31, 2023

MAINT unpin cython in CI config (scikit-learn#27627)

f065246

Co-authored-by: Loïc Estève <loic.esteve@ymail.com>

REDVM pushed a commit to REDVM/scikit-learn that referenced this pull request Nov 16, 2023

MAINT unpin cython in CI config (scikit-learn#27627)

8072420

Co-authored-by: Loïc Estève <loic.esteve@ymail.com>

jjerphan mentioned this pull request Dec 20, 2023

Cython compilation error with Cython>=3 and scikit-learn<1.2 #26858

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

MAINT unpin cython in CI config #27627

MAINT unpin cython in CI config #27627

Uh oh!

ogrisel commented Oct 20, 2023 •

edited

Loading

Uh oh!

ogrisel Oct 20, 2023

Uh oh!

github-actions bot commented Oct 20, 2023 •

edited

Loading

Uh oh!

lesteve commented Oct 20, 2023

Uh oh!

ogrisel commented Oct 20, 2023

Uh oh!

ogrisel commented Oct 20, 2023

Uh oh!

lesteve commented Oct 20, 2023 •

edited

Loading

Uh oh!

ogrisel commented Oct 20, 2023

Uh oh!

ogrisel commented Oct 20, 2023

Uh oh!

ogrisel commented Oct 20, 2023

Uh oh!

lesteve commented Oct 20, 2023

Uh oh!

jjerphan left a comment

Uh oh!

Uh oh!

	# TODO: change for `libcpp.algorithm.move` once Cython 3 is used
	# Introduction in Cython:
	# https://github.com/cython/cython/blob/05059e2a9b89bf6738a7750b905057e5b1e3fe2e/Cython/Includes/libcpp/algorithm.pxd#L47 #noqa
	cdef extern from "<algorithm>" namespace "std" nogil:
	OutputIt move[InputIt, OutputIt](InputIt first, InputIt last, OutputIt d_first) except + #noqa

Uh oh!

MAINT unpin cython in CI config #27627

MAINT unpin cython in CI config #27627

Uh oh!

Conversation

ogrisel commented Oct 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ogrisel Oct 20, 2023

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✔️ Linting Passed

Uh oh!

lesteve commented Oct 20, 2023

Uh oh!

ogrisel commented Oct 20, 2023

Uh oh!

ogrisel commented Oct 20, 2023

Uh oh!

lesteve commented Oct 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ogrisel commented Oct 20, 2023

Uh oh!

ogrisel commented Oct 20, 2023

Uh oh!

ogrisel commented Oct 20, 2023

Uh oh!

lesteve commented Oct 20, 2023

Uh oh!

jjerphan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ogrisel commented Oct 20, 2023 •

edited

Loading

github-actions bot commented Oct 20, 2023 •

edited

Loading

lesteve commented Oct 20, 2023 •

edited

Loading