-
-
Notifications
You must be signed in to change notification settings - Fork 591
Exclude static libraries and tests to reduce zipped Python executables #758
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exclude static libraries and tests to reduce zipped Python executables #758
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change looks safe to me. As far as I can tell, these .a
files are only being included in distributions to be re-consumed by other build processes and can't be useful in a Bazel Python runtime.
There's also a bunch of .tcl
files that may be useless...
Co-authored-by: Jonathon Belotti <jonathon@canva.com>
Yes, that can be removed, but removing modules seems arguable to me. We might want to consider removing $ unzip -l bazel-bin/hello.zip | sort -n | tail -n 6
745004 2010-01-01 00:00 runfiles/python3_10_x86_64-unknown-linux-gnu/lib/python3.10/pydoc_data/topics.py
816725 2010-01-01 00:00 runfiles/python3_10_x86_64-unknown-linux-gnu/lib/python3.10/ensurepip/_bundled/setuptools-58.1.0-py3-none-any.whl
2123599 2010-01-01 00:00 runfiles/python3_10_x86_64-unknown-linux-gnu/lib/python3.10/ensurepip/_bundled/pip-22.0.4-py3-none-any.whl
38861696 2010-01-01 00:00 runfiles/python3_10_x86_64-unknown-linux-gnu/lib/libpython3.10.so
38861696 2010-01-01 00:00 runfiles/python3_10_x86_64-unknown-linux-gnu/lib/libpython3.10.so.1.0
110513272 2381 files The zip file should include either shared library. |
PR Checklist
Please check if your PR fulfills the following requirements:
PR Type
What kind of change does this PR introduce?
What is the current behavior?
Issue Number: N/A
Build artifacts by
bazel build
with the--build_python_zip
option are unnecessary big. The size of the artifacts is at least 90MB (e.g., a zipped python binary which just prints "hello, world"). This is not great when build artifacts are included to docker images. The cause of this size issue is that the hermetic Python toolchain include the static libraries of the Python toolchain and tests for the standard libraries even though majority of users of rules_python don't need them. These files occupy 47MB (52% of an artifact of size 90MB).Repro steps
WORKSPACE
:BUILD
:hello.py
:You can see some static libraries are included in the zip file (we have duplicated
libpython3.10.a
. The file size (uncompressed) of the static library is 38MB):What is the new behavior?
The static libraries and tests for the standard libraries in the Python toolchain are excluded from Python runfiles built by Python rules. Build artifacts by
bazel build
with the--build_python_zip
get smaller. The size of the artifacts is at least 43MB, saving 47MB.Does this PR introduce a breaking change?
Other information