Skip to content

Fix leading forward slashes in RECORD files produced by py_wheel. #789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Aug 14, 2022
Merged

Fix leading forward slashes in RECORD files produced by py_wheel. #789

merged 2 commits into from
Aug 14, 2022

Conversation

UebelAndre
Copy link
Contributor

@UebelAndre UebelAndre commented Aug 13, 2022 •
edited
Loading

PR Checklist

Please check if your PR fulfills the following requirements:

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

PR Type

What kind of change does this PR introduce?

  • Bugfix
  • Feature (please, look at the "Scope of the project" section in the README.md file)
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • CI related changes
  • Documentation content changes
  • Other... Please describe:

What is the current behavior?

Issue Number: closes #745

What is the new behavior?

This fixes broken packages produced by py_wheel.

Does this PR introduce a breaking change?

  • Yes
  • No

Other information

@UebelAndre UebelAndre marked this pull request as ready for review August 13, 2022 20:54
groodt
groodt approved these changes Aug 14, 2022
View reviewed changes
Copy link
Collaborator

@groodt groodt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@groodt groodt merged commit 599ddb8 into bazel-contrib:main Aug 14, 2022
rdesgroppes added a commit to rdesgroppes/bookingcom-buildtools that referenced this pull request Nov 24, 2022
@rdesgroppes
Bump rules_python to fix RECORD in .whl file
6b08f83 
Prior to `rules_python` version 0.11.0, `py_wheel` used to generate
invalid RECORD entries for file paths outside of the current package:
```
Traceback (most recent call last):
  File "[...]/pip_install/extract_wheels/extract_single_wheel.py", line 105, in <module>
    main()
  File "[...]/pip_install/extract_wheels/extract_single_wheel.py", line 93, in main
    bazel.extract_wheel(
  File "[...]/pip_install/extract_wheels/bazel.py", line 364, in extract_wheel
    whl.unzip(directory)
  File "[...]/pip_install/extract_wheels/wheel.py", line 90, in unzip
    installer.install(
  File "[...]/pypi__installer/installer/_core.py", line 96, in install
    for record_elements, stream, is_executable in source.get_contents():
  File "[...]/pypi__installer/installer/sources.py", line 158, in get_contents
    assert record is not None, "In {}, {} is not mentioned in RECORD".format(
AssertionError: In unused_deps_py-0.0.5-py3-none-any.whl, compatibility_tests/v2.5.0/tests/__init__.py is not mentioned in RECORD
```

This was reported through
bazel-contrib/rules_python#745, then fixed by
bazel-contrib/rules_python#789.

`rules_python`'s version gets here conservatively bumped to the
corresponding bugfix version:
[0.11.0](https://github.com/bazelbuild/rules_python/releases/tag/0.11.0)

Practically, the main difference between
unused_deps_py-0.0.5.dist-info/RECORD and
unused_deps_py-0.0.6.dist-info/RECORD consists in leading `/` being
stripped:
```diff
-/compatibility_tests/v2.5.0/tests/__init__.py,sha256=8SXBh8YGxPXGDmFtR_n90wt5xWvc5CEp_L5Pn687VaU,150
[...]
-/compatibility_tests/v2.5.0/tests/google/protobuf/internal/__init__.py,sha256=H-9Le6XWW7Ikwaq9qPneZts4ZitfM2XgeK4x-B5joTA,1867
+compatibility_tests/v2.5.0/tests/__init__.py,sha256=8SXBh8YGxPXGDmFtR_n90wt5xWvc5CEp_L5Pn687VaU,150
[...]
+compatibility_tests/v2.5.0/tests/google/protobuf/internal/__init__.py,sha256=H-9Le6XWW7Ikwaq9qPneZts4ZitfM2XgeK4x-B5joTA,1867
```
and:
```diff
-/google/__init__.py,sha256=8SXBh8YGxPXGDmFtR_n90wt5xWvc5CEp_L5Pn687VaU,150
[...]
-/google/protobuf/wrappers_pb2.py,sha256=xybMLGFD_xtm-GZ0owcWN-44dDkI5oR2U8Tl_ZngwpQ,12473
+google/__init__.py,sha256=8SXBh8YGxPXGDmFtR_n90wt5xWvc5CEp_L5Pn687VaU,150
[...]
+google/protobuf/wrappers_pb2.py,sha256=xybMLGFD_xtm-GZ0owcWN-44dDkI5oR2U8Tl_ZngwpQ,12473
```

Note: with latest version of `rules_python` (0.14.0 as of Nov 24, 2022),
the build would fail on:
> ERROR: $PWD/unused_deps_py/unused_deps_py_lib/BUILD.bazel:5:11: no
> such package '@unused_deps_py_pip_java_manifest//': The repository
> '@unused_deps_py_pip_java_manifest' could not be resolved: Repository
> '@unused_deps_py_pip_java_manifest' is not defined and referenced by
> '//unused_deps_py/unused_deps_py_lib:unused_deps_py_lib'
@rdesgroppes rdesgroppes mentioned this pull request Nov 24, 2022
Merged
rdesgroppes added a commit to rdesgroppes/bookingcom-buildtools that referenced this pull request Nov 24, 2022
@rdesgroppes
fix(unused_deps_py): Bump rules_python to fix whl RECORD
f025d78 
Prior to `rules_python` version 0.11.0, `py_wheel` used to generate
invalid RECORD entries for file paths outside of the current package:
```
Traceback (most recent call last):
  File "[...]/pip_install/extract_wheels/extract_single_wheel.py", line 105, in <module>
    main()
  File "[...]/pip_install/extract_wheels/extract_single_wheel.py", line 93, in main
    bazel.extract_wheel(
  File "[...]/pip_install/extract_wheels/bazel.py", line 364, in extract_wheel
    whl.unzip(directory)
  File "[...]/pip_install/extract_wheels/wheel.py", line 90, in unzip
    installer.install(
  File "[...]/pypi__installer/installer/_core.py", line 96, in install
    for record_elements, stream, is_executable in source.get_contents():
  File "[...]/pypi__installer/installer/sources.py", line 158, in get_contents
    assert record is not None, "In {}, {} is not mentioned in RECORD".format(
AssertionError: In unused_deps_py-0.0.5-py3-none-any.whl, compatibility_tests/v2.5.0/tests/__init__.py is not mentioned in RECORD
```

This was reported through
bazel-contrib/rules_python#745, then fixed by
bazel-contrib/rules_python#789.

`rules_python`'s version gets here conservatively bumped to the
corresponding bugfix version:
[0.11.0](https://github.com/bazelbuild/rules_python/releases/tag/0.11.0)

Practically, the main difference between
unused_deps_py-0.0.5.dist-info/RECORD and
unused_deps_py-0.0.6.dist-info/RECORD consists in leading `/` being
stripped:
```diff
-/compatibility_tests/v2.5.0/tests/__init__.py,sha256=8SXBh8YGxPXGDmFtR_n90wt5xWvc5CEp_L5Pn687VaU,150
[...]
-/compatibility_tests/v2.5.0/tests/google/protobuf/internal/__init__.py,sha256=H-9Le6XWW7Ikwaq9qPneZts4ZitfM2XgeK4x-B5joTA,1867
+compatibility_tests/v2.5.0/tests/__init__.py,sha256=8SXBh8YGxPXGDmFtR_n90wt5xWvc5CEp_L5Pn687VaU,150
[...]
+compatibility_tests/v2.5.0/tests/google/protobuf/internal/__init__.py,sha256=H-9Le6XWW7Ikwaq9qPneZts4ZitfM2XgeK4x-B5joTA,1867
```
and:
```diff
-/google/__init__.py,sha256=8SXBh8YGxPXGDmFtR_n90wt5xWvc5CEp_L5Pn687VaU,150
[...]
-/google/protobuf/wrappers_pb2.py,sha256=xybMLGFD_xtm-GZ0owcWN-44dDkI5oR2U8Tl_ZngwpQ,12473
+google/__init__.py,sha256=8SXBh8YGxPXGDmFtR_n90wt5xWvc5CEp_L5Pn687VaU,150
[...]
+google/protobuf/wrappers_pb2.py,sha256=xybMLGFD_xtm-GZ0owcWN-44dDkI5oR2U8Tl_ZngwpQ,12473
```

Note: with latest version of `rules_python` (0.14.0 as of Nov 24, 2022),
the build would fail on:
> ERROR: $PWD/unused_deps_py/unused_deps_py_lib/BUILD.bazel:5:11: no
> such package '@unused_deps_py_pip_java_manifest//': The repository
> '@unused_deps_py_pip_java_manifest' could not be resolved: Repository
> '@unused_deps_py_pip_java_manifest' is not defined and referenced by
> '//unused_deps_py/unused_deps_py_lib:unused_deps_py_lib'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@groodt groodt groodt approved these changes

@thundergolfer thundergolfer Awaiting requested review from thundergolfer

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

RECORD file is broken if strip_path_prefixes used
2 participants
@UebelAndre @groodt