Skip to content

[MAINT, Cython] Implicit noexcept is deprecated in Cython 3.0 #25609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
45 tasks
adam2392 opened this issue Feb 14, 2023 · 3 comments · Fixed by #25621
Closed
45 tasks

[MAINT, Cython] Implicit noexcept is deprecated in Cython 3.0 #25609

adam2392 opened this issue Feb 14, 2023 · 3 comments · Fixed by #25621
Labels
cython

Comments

@adam2392
Copy link
Member

adam2392 commented Feb 14, 2023 •
edited
Loading

Hi,

I was trying some stuff out on Cython 3.0, and I saw a bunch of errors of the form:

...
  warning: sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors.pyx:954:49: Implicit noexcept declaration is deprecated. Function declaration should contain 'noexcept' keyword.
  warning: sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors.pyx:966:11: Implicit noexcept declaration is deprecated. Function declaration should contain 'noexcept' keyword.
  warning: sklearn/utils/_sorting.pxd:9:7: Implicit noexcept declaration is deprecated. Function declaration should contain 'noexcept' keyword.
  warning: sklearn/utils/_vector_sentinel.pxd:12:60: Implicit noexcept declaration is deprecated. Function declaration should contain 'noexcept' keyword.

It seems that anytime nogil is used, the noexcept keyword must be typed. Is this of interest to patch?

Proposed Solution

It is thus time to replace the following instances in sklearn to be compatible with Cython3.0+:

  • nogil with noexcept nogil and
  • nogil except -1 with except -1 nogil

By running:

ag "nogil:" -c

You can see the following files are affected. The proposed strategy is to then tackle 1 submodule at a time (or one set of files at a time).

  • sklearn/tree/_utils.pyx:16
  • sklearn/tree/_oblique_tree.pyx:2
  • sklearn/tree/_splitter.pyx:31
  • sklearn/tree/_criterion.pyx:35
  • sklearn/tree/_oblique_splitter.pyx:10
  • sklearn/tree/_tree.pyx:19
  • sklearn/metrics/_dist_metrics.pyx.tp:3
  • sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.pyx.tp:17
  • sklearn/metrics/_pairwise_distances_reduction/_base.pyx.tp:14
  • sklearn/metrics/_pairwise_distances_reduction/_datasets_pair.pyx.tp:20
  • sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors.pyx.tp:16
  • sklearn/metrics/_pairwise_distances_reduction/_argkmin.pyx.tp:16
  • sklearn/metrics/_pairwise_fast.pyx:1
  • sklearn/ensemble/_hist_gradient_boosting/histogram.pyx:8
  • sklearn/ensemble/_hist_gradient_boosting/_predictor.pyx:2
  • sklearn/ensemble/_hist_gradient_boosting/_bitset.pyx:5
  • sklearn/ensemble/_hist_gradient_boosting/splitting.pyx:11
  • sklearn/ensemble/_gradient_boosting.pyx:1
  • sklearn/cluster/_k_means_elkan.pyx:2
  • sklearn/cluster/_k_means_common.pyx:2
  • sklearn/cluster/_k_means_minibatch.pyx:2
  • sklearn/cluster/_k_means_lloyd.pyx:2
  • sklearn/_loss/_loss.pyx.tp:37
  • sklearn/_isotonic.pyx:1
  • sklearn/linear_model/_sag_fast.pyx.tp:10
  • sklearn/linear_model/_cd_fast.pyx:10
  • sklearn/linear_model/_sgd_fast.pyx:26
  • sklearn/utils/sparsefuncs_fast.pyx:1
  • sklearn/utils/_random.pxd:1
  • sklearn/utils/_openmp_helpers.pyx:1
  • sklearn/utils/_sorting.pyx:2
  • sklearn/utils/_cython_blas.pyx:11
  • sklearn/utils/_weight_vector.pyx.tp:6
  • sklearn/utils/_isfinite.pyx:4
  • sklearn/utils/_seq_dataset.pyx.tp:8
  • sklearn/utils/_heap.pyx:1
  • sklearn/svm/_liblinear.pyx:1
  • sklearn/svm/_libsvm.pyx:5
  • sklearn/svm/_libsvm_sparse.pyx:3
  • sklearn/manifold/_barnes_hut_tsne.pyx:3
  • sklearn/preprocessing/_csr_polynomial_expansion.pyx:3
  • sklearn/decomposition/_cdnmf_fast.pyx:1
  • sklearn/decomposition/_online_lda_fast.pyx:1
  • sklearn/neighbors/_quad_tree.pyx:6
  • sklearn/neighbors/_binary_tree.pxi:3

You can run specifically ag ") nogil:" -Q, if you want a more fine-grained search.

Once the changes are made, one must run the following:

pip install --upgrade cython
cython --version
> # should be 3.0
pip install --verbose --editable . 2>&1 | grep -C 3 "sklearn.metrics"

and copy/paste the output for each PR to verify that they have indeed made the changes correctly.
@github-actions github-actions bot added the Needs Triage Issue requires triage label Feb 14, 2023
@thomasjpfan
Copy link
Member

Yea, we should do this and follow the advice in the Cython migration guide. The TODOs is listed here:

scikit-learn/sklearn/_build_utils/__init__.py

Lines 83 to 93 in 872a084

# TODO: once Cython 3 is released and we require Cython>=3 we should get
# rid of the `legacy_implicit_noexcept` directive.
# This should mostly consist in:
#
# - ensuring nogil is at the end of function signature,
# e.g. replace "nogil except -1" by "except -1 nogil".
#
# - "noexcept"-qualifying Cython and externalized C interfaces
# which aren't raising nor propagating exceptions.
# See: https://cython.readthedocs.io/en/latest/src/userguide/language_basics.html#error-return-values # noqa
#

If the changes are compatible with 0.29.33 and 3.0, I am in favor of updating the code now.

@thomasjpfan thomasjpfan added cython and removed Needs Triage Issue requires triage labels Feb 14, 2023
@adam2392
Copy link
Member Author

Both replacement of:

  • nogil with noexcept nogil and
  • nogil except -1 with except -1 nogil

are valid for 0.29.33. I can tackle this by doing one submodule at a time perhaps and then c/ping the output of the pip install with Cython3.0? I have updated the issue with a strategy.

@thomasjpfan
Copy link
Member

Given this is a simple find and replace, I think it can be big one PR. Here is a quick Python script that does the replacements:

import re
from pathlib import Path
from itertools import chain

sklearn_root = Path("sklearn")

pyx_paths = chain(
    sklearn_root.glob("**/*.pyx"),
    sklearn_root.glob("**/*.pyx.tp"),
    sklearn_root.glob("**/*.pxi"),
)


nogil_colon = re.compile(r"\) nogil:")
nogil_except_neg1 = re.compile(r"\) nogil except -1:")
nogil_except_star = re.compile(r"\) nogil except \*:")

for pyx_path in pyx_paths:
    orig_contents = pyx_path.read_text()
    new_contents = nogil_colon.sub(") noexcept nogil:", orig_contents)
    new_contents = nogil_except_neg1.sub(") except -1 nogil:", new_contents)
    new_contents = nogil_except_star.sub(") except nogil *:", new_contents)

    if new_contents != orig_contents:
        pyx_path.write_text(new_contents)


nogil_no_colon = re.compile(r"\) nogil\n")
nogil_except_neg1_no_colon = re.compile(r"\) nogil except -1")
nogil_except_star_no_colon = re.compile(r"\) nogil except \*")

pxd_paths = chain(sklearn_root.glob("**/*.pxd"), sklearn_root.glob("**/*.pxd.tp"))

for pxd_path in pxd_paths:
    orig_contents = pxd_path.read_text()
    new_contents = nogil_no_colon.sub(") noexcept nogil\n", orig_contents)
    new_contents = nogil_except_neg1_no_colon.sub(") except -1 nogil", new_contents)
    new_contents = nogil_except_star_no_colon.sub(") except * nogil", new_contents)

    if new_contents != orig_contents:
        pxd_path.write_text(new_contents)

Afterwards, we can remove legacy_implicit_noexcept.

@adam2392 adam2392 mentioned this issue Feb 15, 2023
Merged
@benfred benfred mentioned this issue Jul 19, 2023
Closed
@harryting-yiting harryting-yiting mentioned this issue Sep 9, 2023
Open
@saethlin saethlin mentioned this issue Sep 12, 2023
Merged
@rafaela-amorim rafaela-amorim mentioned this issue Jul 4, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
cython
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

MAINT Remove Cython compilation warnings ahead of Cython3.0 release neurodata/scikit-learn
2 participants
@adam2392 @thomasjpfan