✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: e4fa36d. Link to the linter CI: here}

From the spin side, we are very willing to adapt the tool to best address the projects' use-cases; it is meant to make development easier, not to obfuscate things further!

Perhaps the simplest way to think of it, in context of this project, is that it is a Python tool that replaces the Makefile. So, no more of a "layer cake of tools" than you already have 😅 Because it is a Python tool, however, we can improve the user interface: print a decent help, add flags, etc. Because it is not written in Make, you can put logic in your commands that other developers can understand (spin was written from the outset to allow customization; we provide built-in commands, to make it easier to get going, but we always knew project needs would differ). In the sense of the broader ecosystem, it is convenient to be able to use the same user interface to build numpy, scikit-learn, and several other projects (it started when projects switched to Meson, because suddenly seasoned developers didn't know which command to invoke; and, of course, this could happen in other areas too, such as switching docs from sphinx to mystmd, etc.).

As gladly as we'd support scikit-learn's use cases, the project should use what works best for its purposes; and if that's Make, that's perfectly fine too :)

Let me know if you have any questions on using/customizing/extending the tool.

Looking at the existing Makefile, I think you can aggregate targets into the following spin commands (with flags, to select behaviors)

build
test
docs
lint
tags

One issue I have with spin while working on Numpy is that it doesn't allow for editable installs. Or at least on numpy side, one needs to do either spin build OR pip install -e .... and cannot mix the two.

One issue I have with spin while working on Numpy is that it doesn't allow for editable installs.

This is actually fixed in spin 0.9, released 2 weeks ago, which prompted me to take another look at spin. This is mentioned in my too long PR description, I did not have/take the time to make it shorter 😅

See scientific-python/spin#155 if you are curious about the details. AFAICS this is mostly:

• having a way to detect if package is installed in editable mode
• if installed in non-editable mode set PYTHONPATH when running the tests/python code. if installed in editable mode nothing to do.

@lesteve I saw you mentioning editable install in the OP, but I figured this must be only for "easy" packages, otherwise numpy wouldn't say spin can't do editable installs. But if it works, then that seems nice.

Note that latest spin on conda-forge seems to be 0.8 though.

But if it works, then that seems nice.

It works for scikit-learn in this PR (at least in my quick tests), I am pretty sure the spin PR adding support for editable install was tested in scikit-image. I don't see why it would not work for numpy. Having said that, there may be some limitations that will be discovered when more people try to use spin with editable install. It's hard to make predictions especially about the future as the saying goes 😉

Note that latest spin on conda-forge seems to be 0.8 though.

Yeah I know mentioned in my too long PR description too 😉

I just checked, and spin seems to work fine with NumPy+editable installs. You probably saw the recommendation in their doc/source/building/index.rst, but that notification was written before this feature landed. I've been waiting before some more testing before changing the "official" advice.

While I was more on the -1 side a couple of months ago I since used stuff like pixi and I can clearly see the benefit of having a cross-platform tool. Today, I would genuinely change my mind and go with such a tool.

A separate question unrelated to this PR since @stefanv is around :). I played with pixi in this toy project and I'm wondering if spin would envision something related to environment management. I would see to potential benefit: (i) it allows contributors to start with a working environment known to be working and (ii) the multi-environment can be used for the CIs (as with have our own lock files now) and even with the setup-pixi GHA (https://github.com/prefix-dev/setup-pixi).

Like Guillaume I am one of the people not super in favour of having tools that are "too powerful" provide this abstraction. "too powerful" is of course ambiguous and undefinable - my main point of worry is that using a tool like a Makefile, dev containers, spin, etc allows you to hide complexity. And then layer some more complexity on top (e.g. a Makefile inside a dev container). IMHO the fix for your "build and install my project in editable mode" command being too complicated to type/remember is not to add a layer of abstraction, but instead to work on making it possible to have a simpler command.

However, I am softening to the idea of a tool like spin. Call it a +0. I think if we do adopt it, we should remove some of the other "spin like" tools currently in use (maybe that is just the Makefile?). Just to keep the combinatorial jungle in check, avoiding the potential for "layers on layers" and having to maintain several solutions. I am poorly informed about what "spin like" tools are currently in use, because I type pytest --my-favourite-option --just-saw-on-stackoverflow when I want to run the tests.

My final point: I care about being able to mix and match a tool like spin and "direct invocation" (aka pytest -k "array_api"). The reason I care is partly because I like typing things into my terminal ;) but also because I've spent too much time of my life trying to work out what that custom ./build.sh --clean run-tests -- -k array_api actually does and why it is failing. This includes things like "layers upon layers", looking inside it and finding a 15 line cmake command with 25 flags, etc.

A separate question unrelated to this PR since @stefanv is around :). I played with pixi in this toy project and I'm wondering if spin would envision something related to environment management. I would see to potential benefit: (i) it allows contributors to start with a working environment known to be working and (ii) the multi-environment can be used for the CIs (as with have our own lock files now) and even with the setup-pixi GHA (https://github.com/prefix-dev/setup-pixi).

Our aim so far has been to keep spin as simple as possible. I.e., limited dependencies, and strictly a more user-friendly replacement for the Makefile approach sklearn uses. We also do not currently have any mechanism for storing options (everything is specific via flags on the prompt). I.e., we can't yet "remember" state, such as the current environment.

We could consider adding some lightweight env management, but why not use nox or one of the pixi tools you mention?

my main point of worry is that using a tool like a Makefile, dev containers, spin, etc allows you to hide complexity.

This is my main worry too :) As such, spin (at least for the internal commands we provide) always prints the commands it executes to the terminal.

And then layer some more complexity on top (e.g. a Makefile inside a dev container). IMHO the fix for your "build and install my project in editable mode" command being too complicated to type/remember is not to add a layer of abstraction, but instead to work on making it possible to have a simpler command.

We don't always have control over the command lines of the tools we used, however, so it'd be hard to get all tool developers to provide each project with exactly what it needs so that arcane flags are no longer necessary. The proof of this lies in the existence of Makefiles in numerous projects.

I think a cool example of this just came out of this conversation: scientific-python/spin#192 I wasn't aware of that meson feature!

However, I am softening to the idea of a tool like spin. Call it a +0.

I'll take it 😂 No, seriously, I discourage people from adopting the tool unless it makes obvious sense. Development should be kept simple; we don't need additional mental overhead. If simple means adopting spin, then great, but there's obviously more than one way to do it.

I think if we do adopt it, we should remove some of the other "spin like" tools currently in use (maybe that is just the Makefile?).

I would not keep both around; "there should be one obvious way" and all that.

My final point: I care about being able to mix and match a tool like spin and "direct invocation" (aka pytest -k "array_api").

Another concern we share. We had this problem with non-in-place meson builds (until recently, those didn't exist!) where we couldn't easily launch commands and have them pick up the package. So spin run pytest -k "array_api" was used to set the PYTHONPATH and then execute the command.

In scikit-learn's case, it seems like most developers are happy to use the editable install route exclusively, in which case many of the meson commands are no longer necessary.
When developing, say, NumPy and SciPy, I tend to have a stable NumPy install in my working env (which SciPy picks up), but then I also want to modify NumPy and test my changes in that library. For that use-case, the meson build-install approach is useful, even though I could have solved it with multiple envs as well.

We could consider adding some lightweight env management, but why not use nox or one of the pixi tools you mention?

Basically, while spin and pixi are initiated to solve different problem, one as a "task manager" and the other one as an "environment manager", I'm under the impression that they could eventually meet somewhere.

Basically, I used pixi to define tasks that are equivalent to the spin doc, spin test, etc. So using pixi only for the environment would be a bit wasteful considering what it offers. However, those tasks have to be redefined per project and this is where I see the value of spin because we end-up having a large overlap between all scientific packages.

I kind of already mentioned this point to @wolfv at the last PyConDE but my view might be not reasonable at all :)

I kind of already mentioned this point to @wolfv at the last PyConDE but my view might be not reasonable at all :)

It may be worth having a chat. No need to build the same thing twice, but the pre-packaged commands (or the logic of them) may still be worth shipping, even if we use alternative scaffolding.

Seems like pixi should allow plugins and our stack writes a plugin like spin for it 😁

Thanks for the ping :) @stefanv - would love to chat anytime. I am currently at PyCon US (you too?). Otherwise happy to jump on a call and discuss. Spin looks cool!

Is it possible to make things also work for Windows? Currently spin docs html does not work for me because Makefile does not work on Windows (there is the make.bat instead).

Is it possible to make things also work for Windows? Currently spin docs html does not work for me because Makefile does not work on Windows (there is the make.bat instead).

Spin is tested on Windows. You can have spin invoke a platform platform-specific Makefile, although I already test docs building on Windows using git-bash.

that would be one benefit of pixi - pixi tasks are designed to run on Windows and Unix with the same syntax :)

that would be one benefit of pixi - pixi tasks are designed to run on Windows and Unix with the same syntax :)

I don't think pixi can fix this problem: it's the underlying Sphinx Make file that's not working on Windows.

Not sure it is worth it, but I guess it seems feasible that spin uses make.bat on Windows if it exists (I thought on Windows make is supposed to find make.bat but maybe it does not work in this case because this is run through subprocess not sure at all ...). IIRC sphinx-quickstart generates both Makefile and make.bat.

I quickly checked and at least numpy and matplotlib still have a make.bat for their docs. Maybe most Scientific Python projects have dropped make.bat and advise contributors to use git bash on Windows, WSL, or something else ...

Edit: opened scientific-python/spin#206 to do this which has been merged

Maybe most Scientific Python projects have dropped make.bat and advise contributors to use git bash to Windows, WSL, or something else ...

My sense is that that's the general trend, since you have trouble building the library on a "traditional" Windows shell. But spin is not opinionated about this sort of thing...

To thread the needle towards a fully opt-in approach, I reverted the doc change and only added the spin install, spin test and spin docs command. I also tweaked the PR title.

Instead, having a curated centralized dev tool that ensures that project build related tasks/commands are discoverable, interpretable, and accessible allows contributors to focus their time and energy on actual.

I agree with everything you said. The point that makes me hesitant/push back is that I bet when the documentation was first created a similar argument was/could have been made for creating those docs. Over time they became less ideal, which is expected. Coffee and milk mix "for free" when you put them in the same mug. Entropy is cruel that way :) For me, adding spin increases our amount of maintenance work - we need to work on undoing the effect of time on the docs and on our spin setup. I think it will be worth it. But it doesn't absolve us from having to try and make the docs nicer (again).

To thread the needle towards a fully opt-in approach, I reverted the doc change and only added the spin install, spin test and spin docs command. I also tweaked the PR title.

Is it realistic to remove the Makefile (at some point in the future) and use spin for everything? If we want centralisation and discoverability we should aim to go "all in".

Is it realistic to remove the Makefile (at some point in the future) and use spin for everything? If we want centralisation and discoverability we should aim to go "all in".

I think it is definitely in the back of my mind to remove the Makefile. I am a few weeks after this is merged or so, maybe a bit more?

This reminded me, I added a custom spin clean command to do the equivalent of make clean which was added to avoid some compilation errors when switching from numpy<2 to numpy>=2 in the same environment, see #29413 if you want to know more.

Would it make sense to add an optional dependency to pyproject.toml so pip install .[spin] grabs a pinned, tested version of spin? We try not to break backward compatibility, and if we know sklearn uses it we'll test against it before release; but still perhaps better to be safe.

EDIT: Nevermind, I keep forgetting that you can't tell pip to only install dependencies, a long term annoyance for me.

should default spin install also have a -v? At least for me, I'd certainly be doing spin install -- -v

Great to see this merged 🎉!

should default spin install also have a -v

It has a -v for me, see spin install --help below for me, maybe try updating spin if you don't see it?

❯ spin install --help
Usage: spin install [OPTIONS]
                    [PIP_ARGS]...

  💽 Build and install package using pip.

  By default, the package is installed in editable mode.

  Arguments after `--` are passed through to pip, e.g.:

    spin install -- --no-clean

  would translated to:

    pip install . --no-build-isolation --editable --no-clean

Options:
  --editable / --no-editable      Install in editable mode.  [default: editable]
  -v, --verbose                   Print detailed build output.
  --verbose-import / --no-verbose-import
                                  Meson only: importing an editable install may trigger a build. This
                                  flag determines whether to print that build's output.
                                  [default: verbose-import]
  --help                          Show this message and exit.

And now I realised that spin install was actually added in spin 0.8 by @adam2392 nice 🙏 scientific-python/spin#139

should default spin install also have a -v?

Rereading this maybe you meant "should spin install without arguments default be to use verbose mode"?

I would personally vote for -v by default, since I always want to see some kind of output, just to be reassured that something is happening if it takes a bit long and monitor the progress of the compilation.

Nice, missed that -v, so at least a few less characters to type. But yeah I'd also vote for -v to be the default.

There you go, I opened a PR to do this 😉 scientific-python/spin#225

I think spin should also be mentioned in contributing.rst where we tell contributors to install development dependencies.

5. Install the development dependencies:

   .. prompt:: bash

        pip install pytest pytest-cov ruff mypy numpydoc black==24.3.0

I think spin should also be mentioned in contributing.rst where we tell contributors to install development dependencies.

Indeed eventually spin should be mentioned in the doc. For now, this PR was meant to be minimal in order to give the possibility for early adopters (like me but hopefully others too) to use spin it in their daily work and gather feed-back. @StefanieSenger if you try out spin, your feed-back would be super useful 🙏.

Based on feed-back, we may consider mentioning spin in the doc and presenting it as the recommended way of working with scikit-learn. In other words, I am trying to be cautious, move slowly and avoid pushing spin down people's throat. Very understandably people tend to be annoyed when they feel they are being forced to change their habits for reasons that don't seem good enough to them 😉.

I see, thanks for sharing your visions on this. I might soon be able to provide feedback on spin.