✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 066ad2a. Link to the linter CI: here}

I was trying to work out if 1.3.0 was built with cython 3 or not. I think the answer is that 1.3.0 was not built with cython 3 because 1.3.0 was released on 30th June and cython 3.0 was released on 17th July.

If this is true then I'd wait with merging this until (a reasonable amount) before the next release. That way we can see if the regression(s) get fixed in cython/work-around'ed in scikit-learn. That way we have a chance of not having to go back to an older version.

What do you think?

I was trying to work out if 1.3.0 was built with cython 3 or not. I think the answer is that 1.3.0 was not built with cython 3 because 1.3.0 was released on 30th June and cython 3.0 was released on 17th July.

Yes, that's it. One can check against the feedstock logs (for instance, this one).

If this is true then I'd wait with merging this until (a reasonable amount) before the next release. That way we can see if the regression(s) get fixed in cython/work-around'ed in scikit-learn. That way we have a chance of not having to go back to an older version.

What do you think?

I am OK with either waiting or merging and then reverting this PR.

I am OK with either waiting or merging and then reverting this PR.

Ok, both sound good. Maybe the only unknown is how we keep track of the task of reverting/merging :D

I am gently hijacking this PR to double-check that indeed Cython 3 is the main cause of the CircleCI builds timing out as mentioned in #27086 (comment)

Edit: it seems like Cython 3 is indeed the main cause since on the doc build, examples/ensemble/plot_forest_hist_grad_boosting_comparison.py takes ~1 minute instead of ~8 minutes on main. From this build log:

/examples/ensemble/plot_forest_hist_grad_boosting_comparison.py:                            55.97 sec

Feel free to pursue this PR.

Thinking again, I think we have no benefit in using Cython 3 so I think we better use Cython < 3 generally for now.

Thinking again, I think we have no benefit in using Cython 3 so I think we better use Cython < 3 generally for now.

Yeah I was wondering about that too. The consensus at the meeting today was that it is better to keep some builds with cython>=3 to keep testing it. On the other hand we are testing Cython 3 with the scipy-dev build so 🤷

The main thing I am wondering about is about whether we need to change pyproject.toml? It feels like it complicates the setup since in some CI builds, the installation of scikit-learn will change the Cython version compared to the lock file, which can be confusing. This happens if scikit-learn is installed with pip install -e . as in the case if DISTRIB == 'conda-pip-latest' i.e. the pylatest_pip_openblas_pandas build.

FYI (we were wondering about this at the meeting): pip install --no-build-isolation is not going to care about the versions in pyproject.toml, it will try to use what is already installed in the current environment.

I removed the pyproject.toml related changes making this PR more of a CI one to speed-up some builds, especially the doc build.

I would say/hope that there is a good chance that Cython 3 fixes regressions before the release of scikit-learn 1.4. If it's not the case, I guess we can revisit changing pyproject.toml.

Merging this one, thanks!

There is an approval and I think that other comments amounts to another +1. Further improvements/tweaks can be done in a separate PR.