✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 6f02648. Link to the linter CI: here}

This is a draft to be able to experiment.

I checked all packages in pip_dependencies if the required version is available on the conda channel that is used in this CI setting.

I was unsure and have not moved meson, because it was not explicitly mentioned in this comment and I assume there is a reason for it?

Other dependencies have not defined a version in our pyproject.toml and I have moved them to conda_dependencies regardless if they were available: lightgbm, array-api-strict, pytest-xdist, ninja, jupyterlite-sphinx, and jupyterlite-pyodide-kernel. That means they will probably come in different versions as with pypi and we'll see where this leads, when the CIs have run.

Okay, seems that the version change in array-api-strict, which has no specified version our pyproject.toml, but was 2.4 (edit: no it was 2.3.1) with pip, and would be 2.2 with conda defaults as required by thepylatest_pip_openblas_pandas CI caused some tests to fail. So I think I should move it back to the pip_dependencies.

I have moved array-api-strict back to the pip_dependencies, and now it shows up in the diff of the lock file for the first time (before it didn't which I thought was strange) as:

# pip array-api-strict @ https://files.pythonhosted.org/packages/fe/c7/a97e26083985b49a7a54006364348cf1c26e5523850b8522a39b02b19715/array_api_strict-2.3.1-py3-none-any.whl#sha256=0ca6988be1c82d2f05b6cd44bc7e14cb390555d1455deb50f431d6d0cf468ded
# pip array-api-strict @ https://files.pythonhosted.org/packages/e5/33/cede42b7b866db4b77432889314fc652ecc5cb6988f831ef08881a767089/array_api_strict-2.4-py3-none-any.whl#sha256=1cb20acd008f171ad8cce49589cc59897d8a242d1acf8ce6a61c3d57b61ecd14

That shows that before version 2.3.1 was used and now it is the newest version on pypi 2.4, that was released 6 days ago.
I don't know why it is not version-locked since it is a fast developing library, but let's hope now the CI passes.

Edit: Nice they all passed. 🎉

Now it's only the Windows CI failing, and this is unrelated and coming from main.

This PR updates update_environments_and_lock_files.py to use conda for packages that were installed via pip but are now available on conda.

Do you know why we used pip for these and not conda? Was it because the version we wanted were not available on conda-forge? I don't know the answer but I assume that (like so many things in scikit-learn) it was done for a specific reason. If we know the reason then it is less likely to break something by accident

@betatim: I didn't know this before, but @adrinjalali told me that's a maintanance task we do from time to time. It is necessary because the versions we need were not available (before) on the respective conda channel that is used for this job (some use conda-forge and other use conda defaults channel) - but now they might be and we prefer this.

I also took notes while I checked everything. Let me share these, maybe it helps with reviewing:

We have specific minimal version requirements in our pyproject.toml:

• cython>=3.0.10, and 3.0.11 available on https://anaconda.org/main/cython
• threadpoolctl>=3.1.0, and 3.5.0 available on https://anaconda.org/main/threadpoolctl
• meson-python>=0.17.1, and 0.17.1 available on https://anaconda.org/main/meson-python
• "sphinx>=7.3.7", 8.2.3 available on https://anaconda.org/main/sphinx
• "numpydoc>=1.2.0", 1.7.0 available on https://anaconda.org/main/numpydoc
• "scikit-image>=0.19.0", 0.25.2 avlailable on https://anaconda.org/main/scikit-image
• "pytest>=7.1.2", 8.4.1 available on https://anaconda.org/conda-forge/pytest and 8.3.4 on https://anaconda.org/main/pytest
• "sphinxext-opengraph>=0.9.1", 0.10.0 available on https://anaconda.org/conda-forge/sphinxext-opengraph

We do not have a version specified in pyproject.toml and have thus used the newest version available on pypi:

• lightgbm, available with version 4.6.0 on https://anaconda.org/main/lightgbm
• array-api-strict, available with version 2.2 on https://anaconda.org/main/array-api-strict <-- this broke and I reverted (see above)
• pytest-xdist, available with version 3.6.1 on https://anaconda.org/main/pytest-xdist
• ninja, available with version 1.12.1 on https://anaconda.org/main/ninja
• jupyterlite-sphinx, available with version 0.20.2 on https://anaconda.org/conda-forge/jupyterlite-sphinx
• jupyterlite-pyodide-kernel, available with version 0.6.1 on https://anaconda.org/conda-forge/jupyterlite-pyodide-kernel

Whether I have checked the version from the main ("defaults") channel or the conda-forge channel depends on how the channel in the job was specified in build_tools/update_environments_and_lock_files.py. Did I do all that correctly 🤷 ?

Cool! Thanks for doing the "chopping wood and carrying water" on moving things from PyPI to conda.

As far as I can tell all the moves make sense. I think we should wait for the CI to be working again and run it again. Maybe even using the [cd build] marker in a commit message to run the builds?