Skip to content

Rebase onto main #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 234 commits into from
Closed

Rebase onto main #1

wants to merge 234 commits into from

Conversation

ianpegg-bc
Copy link

@ianpegg-bc ianpegg-bc commented May 11, 2023

Mainly, the intention is to include this PR:
bazel-contrib#999

This gives the option to not define a __test__.py and to instead run test files as individual scripts.

groodt and others added 30 commits July 18, 2022 16:15
A single filegroup is used to aggregate the interpreter files.
Its platform specific regex patterns are not compatible to the flag
`--incompatible_disallow_empty_glob` which forbids any pattern to return
an empty list. For example, on a Linux host the `"*.exe",` will be empty.
* Support "from" imports in the python gazelle plugin.

Fixes #709.

* Test case for pip imports using "from foo import bar".

* Test cases for imports of the form "from foo import bar".

* Remove unnecessary flag (replaced with continue's).

* Make sure from imports work with std modules.

- Add test case with `from __future__ import print_function`.

* Fix indentation error.

- Make sure that `from foo import bar, baz` works.
- Add test case for this.

* Fix error message.

- Keep error the same as before.

Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>
When a Python repository is created, there is a difference between Unix/Windows for the path to the python interpreter.

This patch adds a symlink from the platform specific path to the interpreter.

This means that on either platform the `:python` target will always point to the interpreter.
…es (#779)

Update Python minor toolchain versions to allow smaller Python toolchains

Co-authored-by: Matt Mackay <mattem@gmail.com>
…t-Is-Purelib: true' (#768)

Co-authored-by: Alex Eagle <alex@aspect.dev>
#808)

Revert "Add option to use "pip download" instead of "pip wheel" to download wheels for other platforms (#773)"

This reverts commit 6a43ebd.
* Fix missing python includes for Unix hosts

* Update python/pip_install/pip_repository.bzl

Co-authored-by: Greg Roodt <groodt@gmail.com>
* Make gazelle_python_manifest output deterministic

Fixes 812

* address review comments

Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>
Fixes #802

Co-authored-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com>
aignas and others added 29 commits April 4, 2023 08:47
Bump the buildifier to the latest version in pre-commit and CI at the
same time.

Related to #1148 and #1151.
This apparently was not working to begin with, but the CI running the
example did not catch it because we did not have an empty
`WORKSPACE.bzlmod`
file.

Tested with the CI and with a clean cache on the local laptop.
Since #999, gazelle can
generate multiple `py_test` rules in a single package (when it finds
multiple `*_test.py` or `test_*.py` files and no `__test__.py` file). In
this case, adding new test files to a package with pre-existing
`py_test` rules is not handled properly due to the `MatchAny` property
on the `py_test` kind - it will match the existing `py_test` rule and
edit it instead of adding a new test rule. This PR disables the matching
so that new `py_test` rules are properly generated.
When using the kind `gazelle:map_kind` directive, `gazelle` will
correctly generate the buildfile on the first pass (or if no target of
that type / name are present).

However, when running gazelle a second time (or if a target of the
mapped kind with the same name is present), `gazelle` will error out
saying that it kind create a target of the original kind because a
target of mapped kind is present and has the same name.

Ex:
Given the directive `# gazelle:map_kind py_test py_pytest_test
//src/bazel/rules/python:py_pytest_test.bzl`, `gazelle` will correctly
generate a `py_pytest_test` target where it would have generated a
`py_test` target.

But on a second invocation of `gazelle` (and subsequent invocations) it
will error our with:
```
gazelle: ERROR: failed to generate target "//test/python/common:common_test" of kind "py_test": a target of kind "py_pytest_test" with the same name already exists. Use the '# gazelle:python_test_naming_convention' directive to change the naming convention.
```
The docs explains that for activating coverage support we use
`register_coverage_tool = True` inside the rule
`register_python_toolchains`. There is no such rule, the actual rule
name is `python_register_toolchains`

Signed-off-by: Andrés Felipe Barco Santa <andres@engflow.com>
)

The interface of `repository_ctx.download` and
`repository_ctx.download_and_extract` supports string lists as well as
strings as the value of the `url` argument. This is the ultimate
destination of the `url` attribute in the `tool_versions` dictionary, so
it makes sense for it to support lists as well.

It is often useful to provide multiple download URLs, e.g. when
vendoring deps through a mirror (to guard against issues like [git
archive checksums
changing](https://github.blog/changelog/2023-01-30-git-archive-checksums-may-change/)
while still keeping the canonical download URL) or in an airgapped
setting (to support internal URLs alongside external URLs). This is also
pretty common around Bazel repository rules that download things, e.g.
[http_archive](https://bazel.build/rules/lib/repo/http#http_archive-urls),
so it can be expected to work with `tool_versions` too.
…1173)

The file was removed in Bazel@HEAD in
bazelbuild/bazel#17545
This fixes failures when using rules_python with Bazel@HEAD.

Addresses: bazelbuild/bazel#17874
…1092)

I can never remember what the syntax is of the compare URLs, so just doc
it to make it easier.
Previously when running the compile_pip_requirements test from an
external repository the test failed. This was due to the fact that paths
in the annotations of the lock file contained paths from the external
repository. Another problem was that the requirement files could not be
located.
)

This makes the test pass on Bazel built from head. The failure appears
due to not forcing the test to a Mac platform, so the underlying logic
in `py_test` to detect the Mac platform fails and doesn't add the
ExecutionInfo with the necessary info.

Work towards bazelbuild/bazel/issues/18170
The latest versions of Bazel have removed the
`@bazel_tools//tools/python:private/defs.bzl` file, so it can no longer
be referenced.

Work towards bazelbuild/bazel/issues/18170
This just keeps it at a recent version, which makes errors easier to
grok. rules_testing recently underwent a large refactor, so it's
confusing when errors reference lines that no longer exist in recent
versions.
…execution (#1183)

This is the inverse test of the one testing for the mac platform. As
before, the line forcing the platform to a non-mac platform was
commented out, which meant, when a mac host built it, it would build the
underlying target for Mac, which violated the test's assumptions.

Work towards bazelbuild/bazel/issues/18170

Fixes #1185
…tc (#1190)

It seems that the macros for specifying the requirements break when the
user starts using `incompatible_generate_aliases=True`. This PR fixes
this.

Testing done:
1. Modify the example:
   ```
   $ git diff
diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel
   index ce91228..1750210 100644
   --- a/examples/bzlmod/MODULE.bazel
   +++ b/examples/bzlmod/MODULE.bazel
   @@ -26,6 +26,7 @@ register_toolchains(
    pip = use_extension("@rules_python//python:extensions.bzl", "pip")
    pip.parse(
        name = "pip",
   +    incompatible_generate_aliases=True,
        requirements_lock = "//:requirements_lock.txt",
        requirements_windows = "//:requirements_windows.txt",
    )
   ```
2. Run `bazel build ...` and check that it is still working.

I noticed this when working on #1189 and creating a separate PR for
easier cherry-picking if we wanted to make a patch release which
includes this. I am not sure how I could make an automated test for this
other than creating a separate example.
Change instruction to refer to `MODULE.bazel` instead of
`MODULES.bazel`.
Starlark rules allow giving the tags as a tuple. The helper function
that added the special migration tag assumed tags was always a list,
resulting in an error when it tried to concatenate a list and tuple.

To fix, check if tags is a tuple and concatenate a tuple if so. The
input type of the tags attribute is preserved so that a test verifying
tags can be passed to the underlying rule can be implemented (this test
is to verify there isn't a regression during the rewrite to Starlark).
This fixes a problem where bazel skylib does not load during the
toolchain integration test.

The skylib dependency was introduced by #1191, but skylib was not
present in the WORKSPACE configs of several things. To fix, skylib just
needs to be added to the workspace files.
Having both bzlmod and a WORKSPACE file confuses the user, and I have
#1155 which adds a new example for gazelle and bzlmod.
py_wheel does not propagate "visibility" attribute to the "publish"
rule. The visibility attribute on py_wheel target is not propagated to
the auto-generated "publish" target. This commit adds the visibility
attribute.

Closes: #1192
…text (#1187)

When a string label is passed to the `@rules_testing` analysis_test
functions, the strings are evaluated within the context of
@rules_testing because that is where the actual rule invocation happens.
Without bzlmod, this just requires qualifying the labels with the repo
name (which is what was being done) because there's just a flat global
namespace of repos.

With bzlmod enabled, repo mapping happens, so rules_testing tries to
resolve those repo names using its repo mapping, which doesn't work
because rules_testing's mapping doesn't include every repo using it.
…po (#1206)

It seems that the tool was correctly trying to modify the `.bazelrc` at
the
root of the repo, but the `find` execution would run in the `$PWD`. This
change ensures that the `find` is consistent with the file we are trying
to modify and allows the user to execute the script from anywhere in the
repo.

Tested:
1. Update the deleted packages with the version of the script at HEAD
1. Fix the bug
1. Rerun the script from the 'tests' sub-folder in the repo to ensure
that
   running the script is noop.
1. Revert the changes to '.bazelrc' to reduce conflicts as other PRs are
   modifying it.

Work towards #958.
…rpreter (#1155)

This exposes a new repo and target, `@{name}_host_interpreter//:python`,
created by `python.toolchain()`, that points to the host OS's Python
interpreter for that particular toolchain.

This solves two problems:
1. `pip.parse()` can now refer to the same interpreter used in the
toolchains
2. There is now a canonical, public, way to refer to the host OS Python
interpreter
    for repository rules.

The above were _sort of_ possible for users to do already, but it
required them to write much more configuration and extension code to do
so. This moves that sort of boilerplate into our code so they have a
simpler configuration.

Also:
- removing bzlmod support in the build_file_generation example; making
examples work
  with both WORKSPACE and MODULE is a pain, hence splitting them.
- adding an example of bzlmod and gazelle
- improved documentation in the pip arguments

Closes: #1161
…d in .bazelrc (#1207)

This correctly handles the integration tests and examples that are a
part of
the `rules_python` workspace and should not be included in the deleted
packages
list.

This brings the changes made to the `.bazelrc` very close to what is in
`main`,
but I would like to update this later once #1155 and #1205 are merged.

Fixes #919.
This is used to generate a path, which shouldn't have a trailing
newline.
…uilds easier (#1181)

Normally, Bazel will ignore its convenience symlinks, so putting them in
the .bazelignore file isn't necessary. However, when
`--output_user_root` is set, which is beneficial to set when using
different Bazel versions (it preserves the analysis cache between
versions), the symlinks aren't ignored. Putting them in the bazelignore
file fixes this.
@ianpegg-bc ianpegg-bc closed this May 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.