Skip to content

Gazelle support is not possible with a different package index than the default #574

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
aignas opened this issue Nov 25, 2021 · 0 comments
Closed

Gazelle support is not possible with a different package index than the default #574

aignas opened this issue Nov 25, 2021 · 0 comments

Comments

@aignas
Copy link
Collaborator

aignas commented Nov 25, 2021

🐞 bug report

Sorry about a bug report on the HEAD version of the rules_python, but wanted to share a few findings whilst trying to use it, since I've spent some time tracking this down.

Affected Rule

Gazelle python manifest is affected.

gazelle_python_manifest(
    name = "gazelle_python_manifest",
    modules_mapping = "@modules_map//:modules_mapping.json",
    pip_deps_repository_name = "pip_deps",
    requirements = "//third_party/pip:requirements.txt",
)

And it is not a regresseion

Description

Currently the rule does not allow to use a custom package index.

There are multiple things related to this:

1. It is not possible to define an index other than the default one for the manifest generation

Since the previously linked bazel rule does not have sufficient arguments, the only way to get it sort of working is to use extra definitions in the requirements file as below:

#
# This file is autogenerated by pip-compile
# To update, run:
#
#    bazel run //third_party/pip:requirements.update
#
--extra-index-url https://my.internal.mirror.com

affine==2.3.0 \
    --hash=sha256:2e045def1aa29e613c42e801a7e10e0b9bacfed1a7c6af0cadf8843530a15102 \
    --hash=sha256:34b05b070d954c382e56f02c207a372d8a32621a87653cc30cdd31cd7f65799f
    # via rasterio

2. modules_mapping cannot handle such requirements file

The parser breaks here. It does not expect an extra index line to be present at all and breaks because the version attribute within the wheel struct is not present and this is a bit difficult to debug, because the starlark code assumes that the parsing is successful, but in reality it could break at any line.

3. JSON information is fetched from PyPI only

If a package only exists in a third_party PyPI index, this will fail. I am not sure if this is a problem with the implementation, but for a user who may be using a custom package index, it would be good to know that this is a requirement.

🔬 Minimal Reproduction

Since I am not aware of any publicly available Python PI alternative, I am not sure I can create an example, but I do hope that the analysis helps.

🔥 Exception or Error

INFO: Repository modules_map instantiated at:
  <repo>/src/WORKSPACE:65:16: in <toplevel>
Repository rule modules_mapping defined at:
  /private/.../1df979b2c502142e517abdf16d459cdc/external/rules_python/gazelle/modules_mapping/def.bzl:281:34: in <toplevel>
ERROR: An error occurred during the fetch of repository 'modules_map':
   Traceback (most recent call last):
        File "/private/.../1df979b2c502142e517abdf16d459cdc/external/rules_python/gazelle/modules_mapping/def.bzl", line 46, column 25, in _modules_mapping_impl
                wheels = _get_wheels(rctx, python_interpreter, pythonpath, parsed_requirements)
        File "/private/.../1df979b2c502142e517abdf16d459cdc/external/rules_python/gazelle/modules_mapping/def.bzl", line 176, column 32, in _get_wheels
                requirement.version,
Error: 'struct' value has no field or method 'version'
Available attributes: name
ERROR: Error fetching repository: Traceback (most recent call last):
        File "/private/.../1df979b2c502142e517abdf16d459cdc/external/rules_python/gazelle/modules_mapping/def.bzl", line 46, column 25, in _modules_mapping_impl
                wheels = _get_wheels(rctx, python_interpreter, pythonpath, parsed_requirements)
        File "/private/.../1df979b2c502142e517abdf16d459cdc/external/rules_python/gazelle/modules_mapping/def.bzl", line 176, column 32, in _get_wheels
                requirement.version,
Error: 'struct' value has no field or method 'version'
Available attributes: name

🌍 Your Environment

Operating System: Mac OS X

Output of bazel version:

$ bazel version
Bazelisk version: development
Build label: 4.2.1
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Mon Aug 30 15:24:28 2021 (1630337068)
Build timestamp: 1630337068
Build timestamp as int: 1630337068

Rules_python version: c3b3c7a72b928fe8c50e20a20dc9d8e520ad3ae11edb2fa803c8267b3b73faf3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@aignas