Skip to content

BLD Fix more build dependencies in meson build #28821

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 15, 2024

Conversation

lesteve
Copy link
Member

@lesteve lesteve commented Apr 12, 2024

Improves the situation for #28820. Since I am not able to reproduce the issue, I guess we will just have to wait and see if this kind of issues happen again after this is merged ...

I tested it for the cases that have been seen in the CI by running this kind of commands from an empty Meson build dir:

ninja -C build/cp312 sklearn/cluster/_hdbscan/_linkage.cpython-312-x86_64-linux-gnu.so.p/sklearn/cluster/_hdbscan/_linkage.pyx.c

I also added utils_cython_tree in all the places that cimport something from utils.

git grep -P 'from \.\..*utils.+cimport'

You can also check inputs are defined correctly by looking at the ninja.build file or with this kind of ninja commands:

❯ ninja -C build/cp312 -t inputs sklearn/cluster/_hdbscan/_linkage.cpython-312-x86_64-linux-gnu.so.p/sklearn/cluster/_hdbscan/_linkage.pyx.c
../../sklearn/__init__.py
../../sklearn/_build_utils/tempita.py
../../sklearn/metrics/__init__.py
../../sklearn/metrics/_dist_metrics.pxd.tp
../../sklearn/utils/__init__.py
../../sklearn/utils/_cython_blas.pxd
../../sklearn/utils/_heap.pxd
../../sklearn/utils/_openmp_helpers.pxd
../../sklearn/utils/_random.pxd
../../sklearn/utils/_seq_dataset.pxd.tp
../../sklearn/utils/_sorting.pxd
../../sklearn/utils/_typedefs.pxd
../../sklearn/utils/_vector_sentinel.pxd
../../sklearn/utils/_weight_vector.pxd.tp
/home/lesteve/dev/scikit-learn/sklearn/cluster/_hdbscan/_linkage.pyx
/home/lesteve/micromamba/envs/scikit-learn-dev/bin/python3.12
sklearn/__init__.py
sklearn/metrics/__init__.py
sklearn/metrics/_dist_metrics.pxd
sklearn/utils/__init__.py
sklearn/utils/_cython_blas.pxd
sklearn/utils/_heap.pxd
sklearn/utils/_openmp_helpers.pxd
sklearn/utils/_random.pxd
sklearn/utils/_seq_dataset.pxd
sklearn/utils/_sorting.pxd
sklearn/utils/_typedefs.pxd
sklearn/utils/_vector_sentinel.pxd
sklearn/utils/_weight_vector.pxd

Here you can check that cythonizing sklearn/cluster/_hdbscan/_linkage.pyx does depend on sklearn/__init__.py, sklearn/utils/__init__.py, sklearn/metrics/__init__.py and utils + metrics .pxd files.

Copy link

github-actions bot commented Apr 12, 2024

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: 249b8bd. Link to the linter CI: here

@@ -16,7 +16,7 @@ tree_extension_metadata = {
foreach ext_name, ext_dict : tree_extension_metadata
py.extension_module(
ext_name,
ext_dict.get('sources'),
ext_dict.get('sources') + utils_cython_tree,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From looking at the other changes, py.extension_module accepts a list of list. If that's the case, can we keep how we call py.extension_module consistent:

Suggested change
ext_dict.get('sources') + utils_cython_tree,
[ext_dict.get('sources'), utils_cython_tree],

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I followed your suggestion.

I actually learned very recently that there was such a thing as argument flattening Meson, so you can do a variety of different syntaxes that should do the same including some complicated ones with arbitrary nesting like this:

['some-file.pyx'], [['some-file.pxd'], [[utils_cython_tree]]]]

see https://mesonbuild.com/Syntax.html#argument-flattening for more details.

Copy link
Member

@betatim betatim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets try it.

@betatim betatim merged commit af236f6 into scikit-learn:main Apr 15, 2024
@lesteve lesteve deleted the fix-meson-dependencies branch April 15, 2024 08:51
@lesteve
Copy link
Member Author

lesteve commented Apr 15, 2024

Thanks, let me know if you see variations of these build errors happen again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants