Uhm there is still something forcing sphinx to use 3.2.0

I'm not opposed to merging this but I don't think our current approach for dev dependencies is ideal.

By specifying min version and unbound max version we are mostly sure that, eventually

• we will get a broken CI on main due to sphinx or some plugin making a major release with breaking changes. Then no one will be able to work on their PR until someone fixes it.
• that for a release made now, building the documentation following the instructions will not work after a couple of years again due to breaking changes in dependencies
• generally it's more work to support multiple sphinx versions, and it's unclear what's the benefit

Instead I think it would be better to loosely pin the sphinx version (and all dev dependencies) as for instance sphinx ~= 4.0.1 (that would match >=4.0.1, <4.1.0) which would address the above issues. Also we can then update sphinx when we decide, not when they happen to make a release.

Generally I think that unpinned max version and then complaining that things break is a problem in the Python ecosystem. For instance tools like Poetry have much better defaults in that aspect.

@rth I still think that we should detect a breakage but I agree that noises on the main branch will be annoying. I am not opposed to pin the max version of sphinx but could we add the max version to the nightly build to check what is going with sphinx, in the same manner that we check of scipy?

but could we add the max version to the nightly build to check what is going with sphinx, in the same manner that we check of scipy?

OK, but what problem would we be solving with that? With scipy we indeed want to make sure that current version of scikit-learn works with future (and not yet released) versions of scipy because many users will be in that situation.

With sphinx and sphinx plugins we don't care. There are probably like 3 contributors who build scikit-learn docs locally (personally I mostly rely on CI lately) and so if you tell them you need to install sphinx 4.0.1 exactly, they will install that version. If there is a 5.0.0 release that breaks something for us, we can very easily keep using 4.0.1 until that's resolved in a next minor release, there is really not rush and no pressure. So I'm not sure this is worth a) having extra CI complexity for it b) have occasional CI breakage on main.

I updated the PR to only address #20026 so sphinx is still pinned to the minimum version.

Instead I think it would be better to loosely pin the sphinx version (and all dev dependencies) as for instance

I am +1 for loosely pinning for all dev dependencies. If we want to stay up to date with the dev dependency, we can setup CI to update the pin with some predefined frequency.

Fine but my only fear is that we will still be using sphinx 3.2.0 in a year or so and the upgrading might be painful when required.