🚀 feature request

Relevant Rules

• //python/extensions:python.bzl: python.toolchain
• //python:repositories.bzl: python_register_toolchains

Description

While the underlying rules for python toolchains/runtimes support python stacks which are actual Bazel-built targets passed in by label, python_register_toolchains, and by extension, the python.toolchain tag class, does not offer a way to do this. At present it supports only downloading hermetic toolchains from GitHub by version number. While one can technically do all the behind-the-scenes work that it performs using one's own module/workspace code, what exactly they do and what expectations they have for the repositories one points them at are not documented. Especially as we make more use of additional types of toolchains (cc toolchains, exec tools toolchains), discovering and writing this much code is an increasing burden. We should provide better support for people who supply their own python stacks, either built from source or precompiled.

Describe the solution you'd like

Digging through python_register_toolchains and a bunch of other code, it looks like each platform expects three toolchains, a coverage tool, a config setting, and the seven other targets which toolchain_aliases repositories name. For each case like this, we can either define a contract for what targets the user-supplied repository+package needs to provide (perhaps using some helpful macros or tag classes from us), then ask them to tell us where to fetch it from and how to patch it; or prompt for each of these things individually. Either way, we could do this either by extending the existing python_register_toolchains and py.toolchain, or by defining a new pair of these things dedicated to this use case.

For python stacks built from source, we can allow people to supply just one repository+package to use everywhere and let Bazel do all the platform management for us. I don't know how best to build a good UX around this for python stacks supplied as binaries without forcing a lot of repetition onto people who want to support lots of platforms. But to an extent that work is also self-imposed, and people could use macros or a platform-based, toolchain_aliases-like repository to manage that complexity.

Describe alternatives you've considered

Instead of supporting the in-build use case ourselves, we could document what people need to do in order to do it themselves and still be able to integrate with the python ecosystem correctly. That makes it far less likely that people will overlook things like they would with source code dives, but it does effectively force people to reimplement much of our finicky loading phase code. Most of the smaller bits comprising it aren't exposed publicly, but perhaps opening some of it up would help support that scenario a bit.