Skip to content

fix: Fixing select when we don't have specific platforms #1364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

chrislovecnm
Copy link
Contributor

In certain releases of Python we do not have various platforms like ppc. When using the call PLATFORMS.keys() we get all of the platforms instead of the platforms that are included with the Python version.

We get the error "no such package" because, for instance, python_3_10_ppc64le-unknown-linux-gnu does not exist in Python 3.10.9 (see versions.bzl).

You can recreate this error by running a bazel query command inside the examples/bzlmod directory. If you run:

bazel query "deps(@python_3_10//:python_headers)"

With a version of the rules previous to this commit, you will get an error. With this commit, you get a printout of all of the python_headers for all of the platforms that are supported in the Python 3.10 version used by the repository.

Instead of using PLATFORMS.keys(), we use the sha256 keys in TOOL_VERSIONS dict based on the full Python version. This change now returns only the platforms supported in the Python version.

Fixes: #1326

@chrislovecnm chrislovecnm requested a review from f0rmiga as a code owner August 5, 2023 15:54
@chrislovecnm chrislovecnm force-pushed the platform_keys_fix branch 2 times, most recently from d4721af to 8785634 Compare August 5, 2023 16:03
@chrislovecnm
Copy link
Contributor Author

@rickeylev any idea why we are doing:

alias(name = "python_headers",  actual = select({":" + item: "@python_3_10_" + item + "//:python_headers" for item in TOOL_VERSIONS['3.10.9']['sha256'].keys()}))

Instead of:

alias(name = "python_headers",  actual = "@python_3_10_x86_64-unknown-linux-gnu//:python_headers")

I'm uncertain how to test that this change does not break anything. Here is most of the BUILD.bazel file that is created by this rule:

 alias(name = "files",           actual = select({":" + item: "@python_3_10_" + item + "//:files" for item in TOOL_VERSIONS['3.10.9']['sha256'].keys()}))
 alias(name = "includes",        actual = select({":" + item: "@python_3_10_" + item + "//:includes" for item in TOOL_VERSIONS['3.10.9']['sha256'].keys()}))
 alias(name = "libpython",       actual = select({":" + item: "@python_3_10_" + item + "//:libpython" for item in TOOL_VERSIONS['3.10.9']['sha256'].keys()}))
 alias(name = "py3_runtime",     actual = select({":" + item: "@python_3_10_" + item + "//:py3_runtime" for item in TOOL_VERSIONS['3.10.9']['sha256'].keys()}))
 #alias(name = "python_headers",  actual = select({":" + item: "@python_3_10_" + item + "//:python_headers" for item in TOOL_VERSIONS['3.10.9']['sha256'].keys()}))
 alias(name = "python_headers",  actual = "@python_3_10_x86_64-unknown-linux-gnu//:python_headers")
 alias(name = "python_runtimes", actual = select({":" + item: "@python_3_10_" + item + "//:python_runtimes" for item in TOOL_VERSIONS['3.10.9']['sha256'].keys()}))
 alias(name = "python3",         actual = select({":" + item: "@python_3_10_" + item + "//:" + ("python.exe" if "windows" in item else "bin/python3") for item in TOOL_VERSIONS['3.10.9']

Running:

bazel query "deps(@python_3_10//:python_headers)"

Works but only gives you the headers for specific host platform with the change. With the select statement, we get the headers for every platform.

@chrislovecnm chrislovecnm marked this pull request as draft August 5, 2023 17:15
@chrislovecnm chrislovecnm marked this pull request as ready for review August 5, 2023 17:20
@chrislovecnm chrislovecnm requested review from rickeylev and removed request for f0rmiga August 5, 2023 17:23
@chrislovecnm
Copy link
Contributor Author

Based on a conversation with @rickeylev we need to have a select with all of the PLATFORMS that are supported in it. This PR wins ;)

Comment on lines +156 to +159
full_python_version = rctx.attr.python_version
if TOOL_VERSIONS.get(full_python_version) == None:
full_python_version = MINOR_MAPPING[full_python_version]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had code in #1294 that was similar to this, it was called full_version, which would always ensure that the python version is in X.Y.Z format. I think we could cherry pick it from there. I did the cherry pick in #1340, but that has not been reviewed yet.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And a nit: this could be changed to be full_python_version = MINOR_MAPPING.get(rctx.attr.python_version, rctx.attr.python_version) as we should have an entry in MINOR_MAPPING for each entry in TOOL_VERSIONS.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will wait for #1340 to get in and use that.

In certain releases of Python we do not have various platforms like
ppc. When using the call PLATFORMS.keys() we get all of the
platforms instead of the platforms that are included with the Python
version.

We get the error "no such package" because, for instance,
python_3_10_ppc64le-unknown-linux-gnu does not exist in Python
3.10.9 (see versions.bzl).

You can recreate this error by running a bazel query command inside
the examples/bzlmod directory. If you run:

bazel query "deps(@python_3_10//:python_headers)"

With a version of the rules previous to this commit, you will get
an error. With this commit, you get a printout of all of the
python_headers for all of the platforms that are supported in the
Python 3.10 version used by the repository.

Instead of using PLATFORMS.keys(), we use the sha256 keys in
TOOL_VERSIONS dict based on the full Python version. This change
now returns only the platforms supported in the Python version.

Fixes: bazel-contrib#1326
@chrislovecnm chrislovecnm marked this pull request as draft August 7, 2023 19:17
@chrislovecnm
Copy link
Contributor Author

I will wait for #1340 to get in and use the code to find the full version.

aignas added a commit that referenced this pull request Sep 20, 2023
… X.Y.Z

Before this PR in numerous places we would check the MINOR_MAPPING dict.
This PR adds a function that also fails if the X.Y format is not in the
MINOR_MAPPING dict making the code more robust.

Split from #1340 to unblock #1364.
aignas added a commit that referenced this pull request Sep 27, 2023
… X.Y.Z

Before this PR in numerous places we would check the MINOR_MAPPING dict.
This PR adds a function that also fails if the X.Y format is not in the
MINOR_MAPPING dict making the code more robust.

Split from #1340 to unblock #1364.
github-merge-queue bot pushed a commit that referenced this pull request Sep 27, 2023
… X.Y.Z (#1423)

Before this PR in numerous places we would check the MINOR_MAPPING dict.
This PR adds a function that also fails if the X.Y format is not in the
MINOR_MAPPING dict making the code more robust.

Split from #1340 to unblock #1364.

---------

Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>
@aignas
Copy link
Collaborator

aignas commented Oct 12, 2023

Implemented by #1473 and I have verified that the bug no longer exists. Verified that

bazel query "deps(@python_3_10//:python_headers)"

succeeds in the examples/bzlmod.

@aignas aignas closed this Oct 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error with ppc versions
2 participants