Skip to content

Commit a5a7ffb

Browse files
f0rmigaalexeagleJonathon Belotti
authored
Gazelle extension for Python (bazel-contrib#514)
Gazelle plugin * Add new example to --deleted_packages * Update examples/build_file_generation/BUILD Co-authored-by: Jonathon Belotti <jonathon@canva.com> * fix: gazelle:exclude on coarse-grained Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: comment on Kinds() Co-authored-by: Jonathon Belotti <jonathon@canva.com> * owner: f0rmiga Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: build and setuptools pinned versions With the recent change in pypa/setuptools#2769, some wheels started to fail build immediately with an unpinned setuptools in isolation mode. Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: use local_repository in examples Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * bump: examples Bazel version Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: add missing .gitignore to example Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * refactor: remove python_coarse_grained_generation Also add the python_generation_mode directive. Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: gazelle spam from org_golang_x_tools Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * revert: example .bazelversion Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: simplify std_modules.py Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: test py_library without __init__.py Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * feat: manifest generation tag manual Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> * fix: check std modules last Performing the check last is more correct and yields better performance, noticeable on large repositories. Signed-off-by: Thulio Ferraz Assis <3149049+f0rmiga@users.noreply.github.com> Co-authored-by: Alex Eagle <eagle@post.harvard.edu> Co-authored-by: Jonathon Belotti <jonathon@canva.com>
1 parent 431caac commit a5a7ffb

File tree

389 files changed

+5702
-2
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

389 files changed

+5702
-2
lines changed

‎.bazelrc

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,15 @@
33
# This lets us glob() up all the files inside the examples to make them inputs to tests
44
# (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it)
55
# To update these lines, run tools/bazel_integration_test/update_deleted_packages.sh
6-
build --deleted_packages=examples/legacy_pip_import/boto,examples/legacy_pip_import/extras,examples/legacy_pip_import/helloworld,examples/pip_install,examples/pip_parse,examples/py_import,examples/relative_requirements
7-
query --deleted_packages=examples/legacy_pip_import/boto,examples/legacy_pip_import/extras,examples/legacy_pip_import/helloworld,examples/pip_install,examples/pip_parse,examples/py_import,examples/relative_requirements
6+
build --deleted_packages=examples/build_file_generation,examples/legacy_pip_import/boto,examples/legacy_pip_import/extras,examples/legacy_pip_import/helloworld,examples/pip_install,examples/pip_parse,examples/py_import,examples/relative_requirements
7+
query --deleted_packages=examples/build_file_generation,examples/legacy_pip_import/boto,examples/legacy_pip_import/extras,examples/legacy_pip_import/helloworld,examples/pip_install,examples/pip_parse,examples/py_import,examples/relative_requirements
88

99
test --test_output=errors
10+
11+
# Do NOT implicitly create empty __init__.py files in the runfiles tree.
12+
# By default, these are created in every directory containing Python source code
13+
# or shared libraries, and every parent directory of those directories,
14+
# excluding the repo root directory. With this flag set, we are responsible for
15+
# creating (possibly empty) __init__.py files and adding them to the srcs of
16+
# Python targets as required.
17+
build --incompatible_default_to_explicit_init_py

‎.github/CODEOWNERS

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,9 @@
1010
/python/whl.bzl @thundergolfer @andyscott
1111
/python/requirements.txt @thundergolfer @andyscott
1212

13+
# Directory containing the Gazelle extension and Go code.
14+
/gazelle/ @f0rmiga
15+
1316
# The proposals dir corresponds to the Bazel proposals process, documented
1417
# here: https://bazel.build/designs/index.html
1518
/proposals/ @brandjon @lberki

‎.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,3 +41,8 @@
4141
# vim swap files
4242
*.swp
4343
*.swo
44+
45+
# Go/Gazelle files
46+
# These otherwise match patterns above
47+
!go.mod
48+
!BUILD.out

‎BUILD

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@
1111
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1212
# See the License for the specific language governing permissions and
1313
# limitations under the License.
14+
load("@bazel_gazelle//:def.bzl", "gazelle")
15+
1416
package(default_visibility = ["//visibility:public"])
1517

1618
licenses(["notice"]) # Apache 2.0
@@ -51,3 +53,19 @@ filegroup(
5153
],
5254
visibility = ["//visibility:public"],
5355
)
56+
57+
# Gazelle configuration options.
58+
# See https://github.com/bazelbuild/bazel-gazelle#running-gazelle-with-bazel
59+
# gazelle:prefix github.com/bazelbuild/rules_python
60+
# gazelle:exclude bazel-out
61+
gazelle(name = "gazelle")
62+
63+
gazelle(
64+
name = "update_go_deps",
65+
args = [
66+
"-from_file=go.mod",
67+
"-to_macro=gazelle/deps.bzl%gazelle_deps",
68+
"-prune",
69+
],
70+
command = "update-repos",
71+
)

‎WORKSPACE

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,8 @@ rules_python_internal_deps()
2424
load("//:internal_setup.bzl", "rules_python_internal_setup")
2525

2626
rules_python_internal_setup()
27+
28+
load("//gazelle:deps.bzl", "gazelle_deps")
29+
30+
# gazelle:repository_macro gazelle/deps.bzl%gazelle_deps
31+
gazelle_deps()

‎examples/build_file_generation/.bazelversion

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
4.0.0

‎examples/build_file_generation/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
bazel-*

‎examples/build_file_generation/BUILD

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
load("@bazel_gazelle//:def.bzl", "gazelle")
2+
load("@rules_python//gazelle:def.bzl", "GAZELLE_PYTHON_RUNTIME_DEPS")
3+
load("@rules_python//gazelle/manifest:defs.bzl", "gazelle_python_manifest")
4+
load("@rules_python//python:defs.bzl", "py_library")
5+
6+
# Gazelle python extension needs a manifest file mapping from
7+
# an import to the installed package that provides it.
8+
# This macro produces two targets:
9+
# - //:gazelle_python_manifest.update can be used with `bazel run`
10+
# to recalculate the manifest
11+
# - //:gazelle_python_manifest.test is a test target ensuring that
12+
# the manifest doesn't need to be updated
13+
gazelle_python_manifest(
14+
name = "gazelle_python_manifest",
15+
modules_mapping = "@modules_map//:modules_mapping.json",
16+
pip_deps_repository_name = "pip",
17+
requirements = "//:requirements_lock.txt",
18+
)
19+
20+
# Our gazelle target points to the python gazelle binary.
21+
# This is the simple case where we only need one language supported.
22+
# If you also had proto, go, or other gazelle-supported languages,
23+
# you would also need a gazelle_binary rule.
24+
# See https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.rst#example
25+
gazelle(
26+
name = "gazelle",
27+
data = GAZELLE_PYTHON_RUNTIME_DEPS,
28+
gazelle = "@rules_python//gazelle:gazelle_python_binary",
29+
)
30+
31+
# This rule is auto-generated and managed by Gazelle,
32+
# because it found the __init__.py file in this folder.
33+
py_library(
34+
name = "build_file_generation",
35+
srcs = ["__init__.py"],
36+
visibility = ["//:__subpackages__"],
37+
)

‎examples/build_file_generation/README.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Build file generation with Gazelle
2+
3+
This example shows a project that has Gazelle setup with the rules_python
4+
extension, so that targets like `py_library` and `py_binary` can be
5+
automatically created just by running
6+
7+
```sh
8+
$ bazel run //:gazelle
9+
```
10+
11+
As a demo, try creating a `__main__.py` file in this directory, then
12+
re-run that gazelle command. You'll see that a `py_binary` target
13+
is created in the `BUILD` file.
14+
15+
Or, try importing the `requests` library in `__init__.py`.
16+
You'll see that `deps = ["@pip//pypi__requests"]` is automatically
17+
added to the `py_library` target in the `BUILD` file.
18+
19+
For more information on the behavior of the rules_python gazelle extension,
20+
see the README.md file in the /gazelle folder.

‎examples/build_file_generation/WORKSPACE

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
workspace(name = "build_file_generation_example")
2+
3+
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
4+
5+
######################################################################
6+
# We need rules_go and bazel_gazelle, to build the gazelle plugin from source.
7+
# Setup instructions for this section are at
8+
# https://github.com/bazelbuild/bazel-gazelle#running-gazelle-with-bazel
9+
10+
# Note, you could omit the rules_go dependency, if you have some way to statically
11+
# compile the gazelle binary for your workspace and distribute it to users on all
12+
# needed platforms.
13+
http_archive(
14+
name = "io_bazel_rules_go",
15+
sha256 = "69de5c704a05ff37862f7e0f5534d4f479418afc21806c887db544a316f3cb6b",
16+
urls = [
17+
"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.27.0/rules_go-v0.27.0.tar.gz",
18+
"https://github.com/bazelbuild/rules_go/releases/download/v0.27.0/rules_go-v0.27.0.tar.gz",
19+
],
20+
)
21+
22+
# NB: bazel-gazelle version must be after 18 August 2021
23+
# to include https://github.com/bazelbuild/bazel-gazelle/commit/2834ea4
24+
http_archive(
25+
name = "bazel_gazelle",
26+
sha256 = "0bb8056ab9ed4cbcab5b74348d8530c0e0b939987b0cfe36c1ab53d35a99e4de",
27+
strip_prefix = "bazel-gazelle-2834ea44b3ec6371c924baaf28704730ec9d4559",
28+
urls = [
29+
# No release since March, and we need subsequent fixes
30+
"https://github.com/bazelbuild/bazel-gazelle/archive/2834ea44b3ec6371c924baaf28704730ec9d4559.zip",
31+
],
32+
)
33+
34+
load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")
35+
load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")
36+
37+
go_rules_dependencies()
38+
39+
go_register_toolchains(version = "1.16.5")
40+
41+
gazelle_dependencies()
42+
43+
######################################################################
44+
# Remaining setup is for rules_python
45+
46+
local_repository(
47+
name = "rules_python",
48+
path = "../..",
49+
)
50+
51+
load("@rules_python//python:pip.bzl", "pip_install")
52+
53+
pip_install(
54+
# Uses the default repository name "pip"
55+
requirements = "//:requirements_lock.txt",
56+
)
57+
58+
# The rules_python gazelle extension has some third-party go dependencies
59+
# which we need to fetch in order to compile it.
60+
load("@rules_python//gazelle:deps.bzl", _py_gazelle_deps = "gazelle_deps")
61+
62+
_py_gazelle_deps()
63+
64+
load("@rules_python//gazelle/modules_mapping:def.bzl", "modules_mapping")
65+
66+
# This repository rule fetches the metadata for python packages we
67+
# depend on. That data is required for the gazelle_python_manifest
68+
# rule to update our manifest file.
69+
# To see what this rule does, try `bazel run @modules_map//:print`
70+
modules_mapping(
71+
name = "modules_map",
72+
requirements = "//:requirements_lock.txt",
73+
)

‎examples/build_file_generation/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
print("hello")

‎examples/build_file_generation/gazelle_python.yaml

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
# GENERATED FILE - DO NOT EDIT!
2+
#
3+
# To update this file, run:
4+
# bazel run //:gazelle_python_manifest.update
5+
6+
manifest:
7+
modules_mapping:
8+
certifi: certifi
9+
certifi.__init__: certifi
10+
certifi.__main__: certifi
11+
certifi.core: certifi
12+
chardet: chardet
13+
chardet.__init__: chardet
14+
chardet.big5freq: chardet
15+
chardet.big5prober: chardet
16+
chardet.chardistribution: chardet
17+
chardet.charsetgroupprober: chardet
18+
chardet.charsetprober: chardet
19+
chardet.cli: chardet
20+
chardet.cli.__init__: chardet
21+
chardet.cli.chardetect: chardet
22+
chardet.codingstatemachine: chardet
23+
chardet.compat: chardet
24+
chardet.cp949prober: chardet
25+
chardet.enums: chardet
26+
chardet.escprober: chardet
27+
chardet.escsm: chardet
28+
chardet.eucjpprober: chardet
29+
chardet.euckrfreq: chardet
30+
chardet.euckrprober: chardet
31+
chardet.euctwfreq: chardet
32+
chardet.euctwprober: chardet
33+
chardet.gb2312freq: chardet
34+
chardet.gb2312prober: chardet
35+
chardet.hebrewprober: chardet
36+
chardet.jisfreq: chardet
37+
chardet.jpcntx: chardet
38+
chardet.langbulgarianmodel: chardet
39+
chardet.langcyrillicmodel: chardet
40+
chardet.langgreekmodel: chardet
41+
chardet.langhebrewmodel: chardet
42+
chardet.langhungarianmodel: chardet
43+
chardet.langthaimodel: chardet
44+
chardet.langturkishmodel: chardet
45+
chardet.latin1prober: chardet
46+
chardet.mbcharsetprober: chardet
47+
chardet.mbcsgroupprober: chardet
48+
chardet.mbcssm: chardet
49+
chardet.sbcharsetprober: chardet
50+
chardet.sbcsgroupprober: chardet
51+
chardet.sjisprober: chardet
52+
chardet.universaldetector: chardet
53+
chardet.utf8prober: chardet
54+
chardet.version: chardet
55+
idna: idna
56+
idna.__init__: idna
57+
idna.codec: idna
58+
idna.compat: idna
59+
idna.core: idna
60+
idna.idnadata: idna
61+
idna.intranges: idna
62+
idna.package_data: idna
63+
idna.uts46data: idna
64+
requests: requests
65+
requests.__init__: requests
66+
requests.__version__: requests
67+
requests._internal_utils: requests
68+
requests.adapters: requests
69+
requests.api: requests
70+
requests.auth: requests
71+
requests.certs: requests
72+
requests.compat: requests
73+
requests.cookies: requests
74+
requests.exceptions: requests
75+
requests.help: requests
76+
requests.hooks: requests
77+
requests.models: requests
78+
requests.packages: requests
79+
requests.sessions: requests
80+
requests.status_codes: requests
81+
requests.structures: requests
82+
requests.utils: requests
83+
urllib3: urllib3
84+
urllib3.__init__: urllib3
85+
urllib3._collections: urllib3
86+
urllib3._version: urllib3
87+
urllib3.connection: urllib3
88+
urllib3.connectionpool: urllib3
89+
urllib3.contrib: urllib3
90+
urllib3.contrib.__init__: urllib3
91+
urllib3.contrib._appengine_environ: urllib3
92+
urllib3.contrib._securetransport: urllib3
93+
urllib3.contrib._securetransport.__init__: urllib3
94+
urllib3.contrib._securetransport.bindings: urllib3
95+
urllib3.contrib._securetransport.low_level: urllib3
96+
urllib3.contrib.appengine: urllib3
97+
urllib3.contrib.ntlmpool: urllib3
98+
urllib3.contrib.pyopenssl: urllib3
99+
urllib3.contrib.securetransport: urllib3
100+
urllib3.contrib.socks: urllib3
101+
urllib3.exceptions: urllib3
102+
urllib3.fields: urllib3
103+
urllib3.filepost: urllib3
104+
urllib3.packages: urllib3
105+
urllib3.packages.__init__: urllib3
106+
urllib3.packages.backports: urllib3
107+
urllib3.packages.backports.__init__: urllib3
108+
urllib3.packages.backports.makefile: urllib3
109+
urllib3.packages.six: urllib3
110+
urllib3.packages.ssl_match_hostname: urllib3
111+
urllib3.packages.ssl_match_hostname.__init__: urllib3
112+
urllib3.packages.ssl_match_hostname._implementation: urllib3
113+
urllib3.poolmanager: urllib3
114+
urllib3.request: urllib3
115+
urllib3.response: urllib3
116+
urllib3.util: urllib3
117+
urllib3.util.__init__: urllib3
118+
urllib3.util.connection: urllib3
119+
urllib3.util.proxy: urllib3
120+
urllib3.util.queue: urllib3
121+
urllib3.util.request: urllib3
122+
urllib3.util.response: urllib3
123+
urllib3.util.retry: urllib3
124+
urllib3.util.ssl_: urllib3
125+
urllib3.util.ssltransport: urllib3
126+
urllib3.util.timeout: urllib3
127+
urllib3.util.url: urllib3
128+
urllib3.util.wait: urllib3
129+
pip_deps_repository_name: pip
130+
integrity: 575d259c512b4b80f9923d1623d2aae3038654b731a4e088bf268e01138b6411

‎examples/build_file_generation/requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
requests==2.25.1

‎examples/build_file_generation/requirements_lock.txt

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#
2+
# This file is autogenerated by pip-compile
3+
# To update, run:
4+
#
5+
# pip-compile --generate-hashes --output-file=requirements_lock.txt requirements.txt
6+
#
7+
certifi==2020.12.5 \
8+
--hash=sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c \
9+
--hash=sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830
10+
# via requests
11+
chardet==3.0.4 \
12+
--hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae \
13+
--hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691
14+
# via requests
15+
idna==2.10 \
16+
--hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \
17+
--hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0
18+
# via requests
19+
requests==2.25.1 \
20+
--hash=sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804 \
21+
--hash=sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e
22+
# via -r requirements.txt
23+
urllib3==1.26.5 \
24+
--hash=sha256:753a0374df26658f99d826cfe40394a686d05985786d946fbe4165b5148f5a7c \
25+
--hash=sha256:a7acd0977125325f516bda9735fa7142b909a8d01e8b2e4c8108d0984e6e0098
26+
# via requests

0 commit comments

Comments
 (0)