bug: Coverage Report Shows no valid records found with bootstrap_impl=script #2572

ewianda · 2025-01-22T04:49:05Z

🐞 bug report

Affected Rule

py_test

Is this a regression?

Yes, bug was introduced in #2409

Description

No coverage report is generated for --@rules_python//python/config_settings:bootstrap_impl=script

🔬 Minimal Reproduction

Steps to reproduce the issue:

Clone the repository:

git clone git@github.com:bazelbuild/rules_python.git
cd rules_python/examples/bzlmod

Run the following command to clean and generate a coverage report:

bazelisk clean && \
bazelisk coverage //tests:version_default_test --combined_report=lcov \
    --@rules_python//python/config_settings:bootstrap_impl=script \
    --cache_test_results=no --instrument_test_targets && \
lcov --list bazel-out/_coverage/_coverage_report.dat

Error Observed:

lcov: ERROR: no valid records found in tracefile bazel-out/_coverage/_coverage_report.dat
(use "lcov --ignore-errors empty ..." to bypass this error)

Checkout a specific commit regression:

git checkout 4a55ef4c2363f7ffc1ec9ba511fba7b2547ccefc
bazelisk clean && \
bazelisk coverage //tests:version_default_test --combined_report=lcov \
    --@rules_python//python/config_settings:bootstrap_impl=script \
    --cache_test_results=no --instrument_test_targets && \
lcov --list bazel-out/_coverage/_coverage_report.dat

Output Observed:

                    |Lines       |Functions  |Branches
Filename            |Rate     Num|Rate    Num|Rate     Num
============================================================
[tests/]
version_test.py     | 0.0%      5|    -     0|    -      0
============================================================
                Total:| 0.0%      5|    -     0|    -      0

🔥 Exception or Error

When running the initial command, the following error is observed:

lcov: ERROR: no valid records found in tracefile bazel-out/_coverage/_coverage_report.dat
(use "lcov --ignore-errors empty ..." to bypass this error)

The text was updated successfully, but these errors were encountered:

rickeylev · 2025-01-22T05:35:38Z

Thanks for the report! I think there's two issues.

First, py_executable.bzl isn't expanding the %coverage_tool% string in the site init hook. That's easy to fix. (wip)

However, after fixing that, the coverage report is still empty. Not sure why that would be.

No idea to test this, either. Only thing I can think of is an integration test where a shell script checks the lcov file.

ewianda · 2025-01-22T05:52:25Z

Just checked out main...rickeylev:rules_python:fix.bootstrap.coverage and it seems to be working

ewianda · 2025-01-22T05:53:55Z

~/projects/rules_python/examples/bzlmod (main *=)
$ git diff
diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl
index da7127e0..6330f1a2 100644
--- a/python/private/py_executable.bzl
+++ b/python/private/py_executable.bzl
@@ -557,6 +557,17 @@ def _create_venv(ctx, output_prefix, imports, runtime_details):
     site_init = ctx.actions.declare_file("{}/_bazel_site_init.py".format(site_packages))
     computed_subs = ctx.actions.template_dict()
     computed_subs.add_joined("%imports%", imports, join_with = ":", map_each = _map_each_identity)
+
+    if (ctx.configuration.coverage_enabled and
+        runtime and
+        runtime.coverage_tool):
+        coverage_tool_runfiles_path = "{}/{}".format(
+            ctx.workspace_name,
+            runtime.coverage_tool.short_path,
+        )
+    else:
+        coverage_tool_runfiles_path = ""
+
     ctx.actions.expand_template(
         template = runtime.site_init_template,
         output = site_init,
@@ -564,6 +575,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details):
             "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False",
             "%site_init_runfiles_path%": "{}/{}".format(ctx.workspace_name, site_init.short_path),
             "%workspace_name%": ctx.workspace_name,
+            "%coverage_tool%": coverage_tool_runfiles_path,
         },
         computed_substitutions = computed_subs,
     )
~/projects/rules_python/examples/bzlmod (main *=)
$ bazelisk clean && bazelisk  coverage //tests:version_default_test --combined_report=lcov  --@rules_python//python/config_settings:bootstrap_impl=script   --cache_test_results=no  --instrument_test_targets && lcov --list bazel-out/_coverage/_coverage_report.dat


INFO: Using default value for --instrumentation_filter: "^//tests[/:]".
INFO: Override the above default with --instrumentation_filter
INFO: Analyzed target //tests:version_default_test (141 packages loaded, 5130 targets configured).
INFO: LCOV coverage report is located at /home/ewianda/.cache/bazel/_bazel_ewianda/229007d1af36718f54c73697ae3aff83/execroot/_main/bazel-out/_coverage/_coverage_report.dat
 and execpath is bazel-out/_coverage/_coverage_report.dat
INFO: From Coverage report generation:
Jan 22, 2025 12:52:47 AM com.google.devtools.coverageoutputgenerator.Main getTracefiles
INFO: Found 1 tracefiles.
Jan 22, 2025 12:52:47 AM com.google.devtools.coverageoutputgenerator.Main parseFilesSequentially
INFO: Parsing file bazel-out/k8-fastbuild/testlogs/tests/version_default_test/coverage.dat
Jan 22, 2025 12:52:47 AM com.google.devtools.coverageoutputgenerator.Main getGcovInfoFiles
INFO: No gcov info file found.
Jan 22, 2025 12:52:47 AM com.google.devtools.coverageoutputgenerator.Main getGcovJsonInfoFiles                                                                                                                                                                                                                            INFO: No gcov json file found.
Jan 22, 2025 12:52:47 AM com.google.devtools.coverageoutputgenerator.Main getProfdataFileOrNull
INFO: No .profdata file found.
INFO: Found 1 test target...
Target //tests:version_default_test up-to-date:
  bazel-bin/tests/version_default_test
INFO: Elapsed time: 6.438s, Critical Path: 2.69s
INFO: 22 processes: 16 internal, 5 linux-sandbox, 1 worker.
INFO: Build completed successfully, 22 total actions
//tests:version_default_test                                             PASSED in 0.5s
  /home/ewianda/.cache/bazel/_bazel_ewianda/229007d1af36718f54c73697ae3aff83/execroot/_main/bazel-out/k8-fastbuild/testlogs/tests/version_default_test/coverage.dat                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 Executed 1 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
                      |Lines       |Functions  |Branches
Filename              |Rate     Num|Rate    Num|Rate     Num
============================================================
[tests/]
version_test.py       | 0.0%      5|    -     0|    -      0
============================================================
                Total:| 0.0%      5|    -     0|    -      0

rickeylev · 2025-01-22T17:06:31Z

Thanks for double checking! I forgot --instrument_test_targets. And I was setting some debug vars that might have interfered. Yes, it looks to work!

The script based bootstrap wasn't expanding the coverage template variable, which prevented coverage from activating. This was introduced when it was switched to the venv layout. To fix, expand the `%coverage_tool%` template variable as done elsewhere. Tested manually, per repro instructions in #2572. While I did devise a way to mostly test this without an integration test, it was thwarted by some other bugs. Along the way, improve some of the bootstrap debug output and fix a comment. Fixes #2572

The script based bootstrap wasn't expanding the coverage template variable, which prevented coverage from activating. This was introduced when it was switched to the venv layout. To fix, expand the `%coverage_tool%` template variable as done elsewhere. Tested manually, per repro instructions in bazel-contrib#2572. While I did devise a way to mostly test this without an integration test, it was thwarted by some other bugs. Along the way, improve some of the bootstrap debug output and fix a comment. Fixes bazel-contrib#2572

ewianda changed the title ~~bug: Coverage Report Shows Invalid Records with bootstrap_impl=script~~ bug: Coverage Report Shows no valid records found with bootstrap_impl=script Jan 22, 2025

rickeylev mentioned this issue Jan 23, 2025

fix: make coverage work with bootstrap=script #2574

Merged

aignas closed this as completed in #2574 Jan 23, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

bug: Coverage Report Shows no valid records found with bootstrap_impl=script #2572

bug: Coverage Report Shows no valid records found with bootstrap_impl=script #2572

ewianda commented Jan 22, 2025

rickeylev commented Jan 22, 2025

Uh oh!

ewianda commented Jan 22, 2025

Uh oh!

ewianda commented Jan 22, 2025

Uh oh!

rickeylev commented Jan 22, 2025

Uh oh!

Uh oh!

bug: Coverage Report Shows no valid records found with bootstrap_impl=script #2572

bug: Coverage Report Shows no valid records found with bootstrap_impl=script #2572

Comments

ewianda commented Jan 22, 2025

🐞 bug report

Affected Rule

Is this a regression?

Description

🔬 Minimal Reproduction

🔥 Exception or Error

rickeylev commented Jan 22, 2025

Uh oh!

ewianda commented Jan 22, 2025

Uh oh!

ewianda commented Jan 22, 2025

Uh oh!

rickeylev commented Jan 22, 2025

Uh oh!