Description
🐞 bug report
Sorry about a bug report on the HEAD version of the rules_python, but wanted to share a few findings whilst trying to use it, since I've spent some time tracking this down.
Affected Rule
Gazelle python manifest is affected.
gazelle_python_manifest(
name = "gazelle_python_manifest",
modules_mapping = "@modules_map//:modules_mapping.json",
pip_deps_repository_name = "pip_deps",
requirements = "//third_party/pip:requirements.txt",
)
And it is not a regresseion
Description
Currently the rule does not allow to use a custom package index.
There are multiple things related to this:
1. It is not possible to define an index other than the default one for the manifest generation
Since the previously linked bazel rule does not have sufficient arguments, the only way to get it sort of working is to use extra definitions in the requirements file as below:
#
# This file is autogenerated by pip-compile
# To update, run:
#
# bazel run //third_party/pip:requirements.update
#
--extra-index-url https://my.internal.mirror.com
affine==2.3.0 \
--hash=sha256:2e045def1aa29e613c42e801a7e10e0b9bacfed1a7c6af0cadf8843530a15102 \
--hash=sha256:34b05b070d954c382e56f02c207a372d8a32621a87653cc30cdd31cd7f65799f
# via rasterio2. modules_mapping cannot handle such requirements file
The parser breaks here. It does not expect an extra index line to be present at all and breaks because the version attribute within the wheel struct is not present and this is a bit difficult to debug, because the starlark code assumes that the parsing is successful, but in reality it could break at any line.
3. JSON information is fetched from PyPI only
If a package only exists in a third_party PyPI index, this will fail. I am not sure if this is a problem with the implementation, but for a user who may be using a custom package index, it would be good to know that this is a requirement.
🔬 Minimal Reproduction
Since I am not aware of any publicly available Python PI alternative, I am not sure I can create an example, but I do hope that the analysis helps.
🔥 Exception or Error
INFO: Repository modules_map instantiated at:
<repo>/src/WORKSPACE:65:16: in <toplevel>
Repository rule modules_mapping defined at:
/private/.../1df979b2c502142e517abdf16d459cdc/external/rules_python/gazelle/modules_mapping/def.bzl:281:34: in <toplevel>
ERROR: An error occurred during the fetch of repository 'modules_map':
Traceback (most recent call last):
File "/private/.../1df979b2c502142e517abdf16d459cdc/external/rules_python/gazelle/modules_mapping/def.bzl", line 46, column 25, in _modules_mapping_impl
wheels = _get_wheels(rctx, python_interpreter, pythonpath, parsed_requirements)
File "/private/.../1df979b2c502142e517abdf16d459cdc/external/rules_python/gazelle/modules_mapping/def.bzl", line 176, column 32, in _get_wheels
requirement.version,
Error: 'struct' value has no field or method 'version'
Available attributes: name
ERROR: Error fetching repository: Traceback (most recent call last):
File "/private/.../1df979b2c502142e517abdf16d459cdc/external/rules_python/gazelle/modules_mapping/def.bzl", line 46, column 25, in _modules_mapping_impl
wheels = _get_wheels(rctx, python_interpreter, pythonpath, parsed_requirements)
File "/private/.../1df979b2c502142e517abdf16d459cdc/external/rules_python/gazelle/modules_mapping/def.bzl", line 176, column 32, in _get_wheels
requirement.version,
Error: 'struct' value has no field or method 'version'
Available attributes: name
🌍 Your Environment
Operating System: Mac OS X
Output of bazel version:
$ bazel version
Bazelisk version: development
Build label: 4.2.1
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Mon Aug 30 15:24:28 2021 (1630337068)
Build timestamp: 1630337068
Build timestamp as int: 1630337068
Rules_python version: c3b3c7a72b928fe8c50e20a20dc9d8e520ad3ae11edb2fa803c8267b3b73faf3