Skip to content

refactor: Updating example to only use bzlmod and updates to example #1147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

refactor: Updating example to only use bzlmod and updates to example #1147

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9fa8034
Updating example to only use bzlmod
chrislovecnm Apr 3, 2023
8f2865f
This is a commit that will fall off this PR
chrislovecnm Apr 3, 2023
2a97c03
Refactoring test name
chrislovecnm Apr 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 0 additions & 27 deletions .bazelci/presubmit.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,33 +119,6 @@ tasks:
test_flags:
- "--test_tag_filters=-integration-test,-acceptance-test"

integration_test_build_file_generation_ubuntu_minimum_supported:
<<: *minimum_supported_version
<<: *reusable_build_test_all
name: build_file_generation integration tests on Ubuntu using minimum supported Bazel version
working_directory: examples/build_file_generation
platform: ubuntu2004
integration_test_build_file_generation_ubuntu:
<<: *reusable_build_test_all
name: build_file_generation integration tests on Ubuntu
working_directory: examples/build_file_generation
platform: ubuntu2004
integration_test_build_file_generation_debian:
<<: *reusable_build_test_all
name: build_file_generation integration tests on Debian
working_directory: examples/build_file_generation
platform: debian11
integration_test_build_file_generation_macos:
<<: *reusable_build_test_all
name: build_file_generation integration tests on macOS
working_directory: examples/build_file_generation
platform: macos
integration_test_build_file_generation_windows:
<<: *reusable_build_test_all
name: build_file_generation integration tests on Windows
working_directory: examples/build_file_generation
platform: windows

integration_test_build_file_generation_bzlmod_ubuntu:
<<: *minimum_supported_bzlmod_version
<<: *common_bzlmod_flags
Expand Down
1 change: 1 addition & 0 deletions examples/build_file_generation/.bazelrc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
common --experimental_enable_bzlmod
test --test_output=errors

# Windows requires these for multi-python support:
Expand Down
3 changes: 2 additions & 1 deletion examples/build_file_generation/BUILD.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ load("@rules_python_gazelle_plugin//:def.bzl", "GAZELLE_PYTHON_RUNTIME_DEPS")
load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest")
load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping")

# This stanza calls a rule that generates targets for managing pip dependencies
# with pip-compile.
compile_pip_requirements(
name = "requirements",
extra_args = ["--allow-unsafe"],
Expand All @@ -21,7 +23,6 @@ compile_pip_requirements(
# This repository rule fetches the metadata for python packages we
# depend on. That data is required for the gazelle_python_manifest
# rule to update our manifest file.
# To see what this rule does, try `bazel run @modules_map//:print`
modules_mapping(
name = "modules_map",
exclude_patterns = [
Expand Down
47 changes: 40 additions & 7 deletions examples/build_file_generation/MODULE.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,43 +1,76 @@
# This file replaces the WORKSPACE file when using bzlmod.

# Declares certain properties of the Bazel module represented by the current Bazel repo.
# These properties are either essential metadata of the module (such as the name and version),
# or affect behavior of the current module and its dependents.
module(
name = "example_bzlmod",
name = "build_file_generation",
version = "0.0.0",
compatibility_level = 1,
)

bazel_dep(name = "rules_python", version = "0.19.0")
bazel_dep(name = "rules_python_gazelle_plugin", version = "0.19.0")
bazel_dep(name = "gazelle", version = "0.29.0", repo_name = "bazel_gazelle")
# The following stanza defines the dependency rules_python.
# For typical setups you set the version.
bazel_dep(name = "rules_python", version = "0.0.0")

# local overrides for the packages for CI purposes.
# for usual setups you should remove this block.
# The following starlark loads rules_python from the file system.
# For usual setups you should remove this local_path_override block.
local_path_override(
module_name = "rules_python",
path = "../..",
)

# The following stanza defines the dependency rules_python.
# For typical setups you set the version.
bazel_dep(name = "rules_python_gazelle_plugin", version = "0.0.0")

# The following starlark loads the gazelle plugingfrom the file system.
# For usual setups you should remove this local_path_override block.
local_path_override(
module_name = "rules_python_gazelle_plugin",
path = "../../gazelle",
)

# Register python toolchain
# The following stanza defines the dependency rules_python.
bazel_dep(name = "gazelle", version = "0.29.0", repo_name = "bazel_gazelle")

# The following stanze Returns a proxy object representing a module extension;
# its methods can be invoked to create module extension tags.
python = use_extension("@rules_python//python:extensions.bzl", "python")

# Using the module extension we register a hermetic Python interpreter rather than relying on
# a system-installed interpreter.
# This toolchain will allow bazel to download a specific python version, and use that version
# for compilation.
python.toolchain(
name = "python3_9",
python_version = "3.9",
)

# Import the python toolchain generated by the given module extension into the scope of the current module.
use_repo(python, "python3_9_toolchains")

# Register an already-defined toolchain so that Bazel can use it during toolchain resolution.
register_toolchains(
"@python3_9_toolchains//:all",
)

# Use the pip extension
pip = use_extension("@rules_python//python:extensions.bzl", "pip")

# Use the extension to call the `pip_repository` rule that invokes `pip`, with `incremental` set.
# Accepts a locked/compiled requirements file and installs the dependencies listed within.
# Those dependencies become available in a generated `requirements.bzl` file.
# You can instead check this `requirements.bzl` file into your repo.
# Because this project has different requirements for windows vs other
# operating systems, we have requirements for each.
pip.parse(
name = "pip",
# Generate user friendly alias labels for each dependency that we have.
incompatible_generate_aliases = True,
requirements_lock = "//:requirements_lock.txt",
requirements_windows = "//:requirements_windows.txt",
)

# Imports the pip toolchain generated by the given module extension into the scope of the current module.
use_repo(pip, "pip")
8 changes: 6 additions & 2 deletions examples/build_file_generation/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Build file generation with Gazelle

This example shows a project that has Gazelle setup with the rules_python
extension, so that targets like `py_library` and `py_binary` can be
automatically created just by running
extension, so that targets like `py_library`, `py_binary`, and `py_test` can be
automatically created just by running:

```sh
$ bazel run //:gazelle
Expand All @@ -18,3 +18,7 @@ added to the `py_library` target in the `BUILD` file.

For more information on the behavior of the rules_python gazelle extension,
see the README.md file in the /gazelle folder.

This example only supports using bzlmod. You can find an older copy of this example
[here](https://github.com/bazelbuild/rules_python/tree/0.20.0/examples/build_file_generation)
that supports not using bzlmod.
121 changes: 0 additions & 121 deletions examples/build_file_generation/WORKSPACE
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,121 +0,0 @@
# Set the name of the bazel workspace.
workspace(name = "build_file_generation_example")

# Load the http_archive rule so that we can have bazel download
# various rulesets and dependencies.
# The `load` statement imports the symbol for http_archive from the http.bzl
# file. When the symbol is loaded you can use the rule.
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

######################################################################
# We need rules_go and bazel_gazelle, to build the gazelle plugin from source.
# Setup instructions for this section are at
# https://github.com/bazelbuild/bazel-gazelle#running-gazelle-with-bazel
# You may need to update the version of the rule, which is listed in the above
# documentation.
######################################################################

# Define an http_archive rule that will download the below ruleset,
# test the sha, and extract the ruleset to you local bazel cache.
http_archive(
name = "io_bazel_rules_go",
sha256 = "099a9fb96a376ccbbb7d291ed4ecbdfd42f6bc822ab77ae6f1b5cb9e914e94fa",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.35.0/rules_go-v0.35.0.zip",
"https://github.com/bazelbuild/rules_go/releases/download/v0.35.0/rules_go-v0.35.0.zip",
],
)

# Download the bazel_gazelle ruleset.
http_archive(
name = "bazel_gazelle",
sha256 = "448e37e0dbf61d6fa8f00aaa12d191745e14f07c31cabfa731f0c8e8a4f41b97",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.28.0/bazel-gazelle-v0.28.0.tar.gz",
"https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.28.0/bazel-gazelle-v0.28.0.tar.gz",
],
)

# Load rules_go ruleset and expose the toolchain and dep rules.
load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")
load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

# go_rules_dependencies is a function that registers external dependencies
# needed by the Go rules.
# See: https://github.com/bazelbuild/rules_go/blob/master/go/dependencies.rst#go_rules_dependencies
go_rules_dependencies()

# go_rules_dependencies is a function that registers external dependencies
# needed by the Go rules.
# See: https://github.com/bazelbuild/rules_go/blob/master/go/dependencies.rst#go_rules_dependencies
go_register_toolchains(version = "1.19.4")

# The following call configured the gazelle dependencies, Go environment and Go SDK.
gazelle_dependencies()

# Remaining setup is for rules_python.

# DON'T COPY_PASTE THIS.
# Our example uses `local_repository` to point to the HEAD version of rules_python.
# Users should instead use the installation instructions from the release they use.
# See https://github.com/bazelbuild/rules_python/releases
local_repository(
name = "rules_python",
path = "../..",
)

local_repository(
name = "rules_python_gazelle_plugin",
path = "../../gazelle",
)

# Next we load the toolchain from rules_python.
load("@rules_python//python:repositories.bzl", "python_register_toolchains")

# We now register a hermetic Python interpreter rather than relying on a system-installed interpreter.
# This toolchain will allow bazel to download a specific python version, and use that version
# for compilation.
python_register_toolchains(
name = "python39",
python_version = "3.9",
)

# Load the interpreter and pip_parse rules.
load("@python39//:defs.bzl", "interpreter")
load("@rules_python//python:pip.bzl", "pip_parse")

# This macro wraps the `pip_repository` rule that invokes `pip`, with `incremental` set.
# Accepts a locked/compiled requirements file and installs the dependencies listed within.
# Those dependencies become available in a generated `requirements.bzl` file.
# You can instead check this `requirements.bzl` file into your repo.
pip_parse(
name = "pip",
# Generate user friendly alias labels for each dependency that we have.
incompatible_generate_aliases = True,
# (Optional) You can provide a python_interpreter (path) or a python_interpreter_target (a Bazel target, that
# acts as an executable). The latter can be anything that could be used as Python interpreter. E.g.:
# 1. Python interpreter that you compile in the build file.
# 2. Pre-compiled python interpreter included with http_archive.
# 3. Wrapper script, like in the autodetecting python toolchain.
#
# Here, we use the interpreter constant that resolves to the host interpreter from the default Python toolchain.
python_interpreter_target = interpreter,
# Set the location of the lock file.
requirements_lock = "//:requirements_lock.txt",
requirements_windows = "//:requirements_windows.txt",
)

# Load the install_deps macro.
load("@pip//:requirements.bzl", "install_deps")

# Initialize repositories for all packages in requirements_lock.txt.
install_deps()

# The rules_python gazelle extension has some third-party go dependencies
# which we need to fetch in order to compile it.
load("@rules_python_gazelle_plugin//:deps.bzl", _py_gazelle_deps = "gazelle_deps")

# See: https://github.com/bazelbuild/rules_python/blob/main/gazelle/README.md
# This rule loads and compiles various go dependencies that running gazelle
# for python requirements.
_py_gazelle_deps()
3 changes: 1 addition & 2 deletions examples/build_file_generation/random_number_generator/BUILD.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,7 @@ py_library(

py_test(
name = "random_number_generator_test",
srcs = ["__test__.py"],
srcs = ["random_number_generator_test.py"],
imports = [".."],
main = "__test__.py",
deps = [":random_number_generator"],
)
0 ...ation/random_number_generator/__test__.py → ...generator/random_number_generator_test.py
View file
Open in desktop
File renamed without changes.
4 changes: 2 additions & 2 deletions python/pip_install/pip_repository.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -370,11 +370,11 @@ def _pip_repository_bzlmod_impl(rctx):
rctx.file("BUILD.bazel", build_contents)
rctx.template("requirements.bzl", rctx.attr._template, substitutions = {
"%%ALL_REQUIREMENTS%%": _format_repr_list([
"@@{}//{}".format(repo_name, p) if rctx.attr.incompatible_generate_aliases else "@{}_{}//:pkg".format(rctx.attr.name, p)
"@{}//{}".format(repo_name, p) if rctx.attr.incompatible_generate_aliases else "@{}_{}//:pkg".format(rctx.attr.name, p)
for p in bzl_packages
]),
"%%ALL_WHL_REQUIREMENTS%%": _format_repr_list([
"@@{}//{}:whl".format(repo_name, p) if rctx.attr.incompatible_generate_aliases else "@{}_{}//:whl".format(rctx.attr.name, p)
"@{}//{}:whl".format(repo_name, p) if rctx.attr.incompatible_generate_aliases else "@{}_{}//:whl".format(rctx.attr.name, p)
for p in bzl_packages
]),
"%%NAME%%": rctx.attr.name,
Expand Down