BLD Improve make clean for edge case situations #29413
Conversation
Co-authored-by: Tim Head <betatim@gmail.com>
| @@ -1,6 +1,7 @@ | |||
| # simple makefile to simplify repetitive build env management tasks under posix | |||
|
|
|||
| PYTHON ?= python | |||
| DEFAULT_MESON_BUILD_DIR = build/cp$(shell python -c 'import sys; print(f"{sys.version_info.major}{sys.version_info.minor}")' ) | |||
There was a problem hiding this comment.
why not just remove build/ ? It's not like the build takes hours.
There was a problem hiding this comment.
In principle you can have different build folders under build and this does not make too much sense to delete them all.
Personally, these days I have many different scikit-learn environments that I create from lock-file to try to reproduce CI issues (especially this week 😓) so I would not want to delete them all and recompile from scratch when I go back to this environmen. Right now I have 3 folders build/cp39, build/cp311, build/cp312.
There was a problem hiding this comment.
In your setup would you get weird error messages in the other envs if you deleted build/ or would it "just" rebuild?
For me it is fine to only delete one sub-directory, but I'd also be happy to just remove all of build/
There was a problem hiding this comment.
You get error message like this if you delete your build folder:
❯ python -c 'import sklearn'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 1262, in _find_spec
File "/home/lesteve/micromamba/envs/testenv/lib/python3.12/site-packages/_scikit_learn_editable_loader.py", line 310, in find_spec
tree = self._rebuild()
^^^^^^^^^^^^^^^
File "/home/lesteve/micromamba/envs/testenv/lib/python3.12/site-packages/_scikit_learn_editable_loader.py", line 338, in _rebuild
if self._work_to_do(env):
^^^^^^^^^^^^^^^^^^^^^
File "/home/lesteve/micromamba/envs/testenv/lib/python3.12/site-packages/_scikit_learn_editable_loader.py", line 325, in _work_to_do
p = subprocess.run(dry_run_build_cmd, cwd=self._build_path, env=env, capture_output=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lesteve/micromamba/envs/testenv/lib/python3.12/subprocess.py", line 548, in run
with Popen(*popenargs, **kwargs) as process:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/lesteve/micromamba/envs/testenv/lib/python3.12/subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/home/lesteve/micromamba/envs/testenv/lib/python3.12/subprocess.py", line 1955, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/lesteve/scikit-learn/build/cp312'
So this would not be ideal:
- env1 is broken let's
rm -rf buildand domake devto rebuild from scratch - OK in the mean time, let's switch to env2 and now work on this other thing
- ah OK weird error in env2 since I
rm -rf buildah I did not think of this 😓 - env2 needs to be rebuilt too oh well one of this day ...
I don't think there is any downside, I have certainly been removing If you are a super experienced meson/ninja user and know how to fix this kind of things:
|
Build systems can't be trusted |
|
I'll merge this now. If Ralf has wisdom to add that leads to us changing our mine we can make a new PR. |
It mostly works in principle, but it's not really possible to guarantee it when a dependency changes as much as between Basically, if header content in numpy changes, The problem here though is that and in numpy 1.26, the So I think a |
Co-authored-by: Tim Head <betatim@gmail.com>
So it turns out a better
make cleanis needed as was discussed around #29296 (comment)When using meson editable mode and switching between numpy<2 and numpy>=2 in the same environment, you can get weird build errors and the way to get out of it is to delete the meson build folder
Removing the default meson build folder
meson/cp312if you are using Python 3.12 seems to be the simplest solution. I triedninja clean -C build/cp312which is a softer clean (my personal newbie understanding).cc @rgommers for visibility or in case off the top of your head this seems like something that should just work in principle. This is completely fine to use this work-around to be honest. It could well be the case that there are some issues with our
meson.buildfiles, hard to tell ...I am able to reproduce with a snippet like this
With the output:
@betatim likely bumped into a similar issue but the compilation error was different: