Skip to content

fix(pypi): make the URL/filename extraction from requirement more robust #2871

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 12, 2025

Conversation

aignas
Copy link
Collaborator

@aignas aignas commented May 11, 2025

Summary:

  • Make the requirement line the same as the one that is used in whls. It
    only contains extras and the version if it is present.
  • Add debug log statements if we fail to get the version from a direct
    URL reference.
  • Move some tests from parse_requirements_tests to
    index_sources_tests to improve test
    maintenance.
  • Replace the URL encoded + to a regular + in the filename.
  • Correctly handle the case when the =sha256: is used in the URL.

Once this is merged I plan to tackle #2648 by changing the
parse_requirements code to de-duplicate entries returned
by the parse_requirements function.

I cannot think of anything else that we can do for this as of now, so
will mark the associated issue as resolved.

Fixes #2363
Work towards #2648

Summary:
- Make the requirement line the same as the one that is used in whls. It
  only contains extras and the version if it is present.
- Add debug log statements if we fail to get the version from a direct
  URL reference.
- Move some tests from `parse_requirements_tests` to
  `index_sources_tests` to improve test
  maintenance.
- Replace the URL encoded `+` to a regular `+` in the filename.
- Correctly handle the case when the `=sha256:` is used in the URL.

I cannot think of anything else that we can do for this as of now, so
will mark the associated issue as resolved.

Fixes bazel-contrib#2363
@aignas aignas requested review from rickeylev and groodt as code owners May 11, 2025 05:47
@rickeylev rickeylev added this pull request to the merge queue May 12, 2025
Merged via the queue into bazel-contrib:main with commit c383c3b May 12, 2025
3 checks passed
@aignas aignas deleted the fix/2363/improve-url-parsing branch May 14, 2025 04:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[pypi] Support pulling dependencies using direct url specifiers without using an index
2 participants