Skip to content

instrument-coverage doesn't quite work #3452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jtracey opened this issue Apr 27, 2022 · 2 comments
Closed

instrument-coverage doesn't quite work #3452

jtracey opened this issue Apr 27, 2022 · 2 comments
Labels

Comments

@jtracey
Copy link
Contributor

jtracey commented Apr 27, 2022

One of the things left to future work in #3396 was code coverage for Android. As described in #3396 (comment), there were some issues preventing that at the time. One of the main points is that Rust on Android is installed via a package manager rather than rustup, so does not have access to nightly. However, as mentioned in that comment, stable rust does now support -C instrument-coverage, which is apparently supposed to be more accurate anyway.

I was able to get this working on Android with approximately the following:

echo "building objects"
objects=$(RUSTFLAGS="-C instrument-coverage" \
            cargo test --no-run --features "$features" --message-format=json \
                | jq -r "select(.profile.test == true) | .filenames[]" \
                | grep -v dSYM -)

echo "running tests"
RUSTFLAGS="-C instrument-coverage" \
         LLVM_PROFILE_FILE="$this_repo/profiles/profile-%m.profraw" \
         cargo test --features "$features" >tests.log

echo "merging profiles"
llvm-profdata merge \
              -sparse \
              "$this_repo/profiles/"*.profraw \
              -o "$this_repo/profiles/combined.profdata"

echo "exporting lcov"
llvm-cov export \
         --ignore-filename-regex='/.cargo/registry' \
         --format=lcov \
         --instr-profile="$this_repo/profiles/combined.profdata" \
         $(for file in $objects; do echo "-object $file"; done) \
         >"$this_repo/profiles/lcov.info"

However, once uploading to codecov, this had the problem of not ignoring #[cfg()]] statements or blank lines. I couldn't see any way of fixing this in the docs for any of the respective llvm tools, so I tried combining it with grcov. I ran basically the commands used currently, just with an extra -b target/debug/ arg, which according to the grcov docs should work. Instead, it just generates an essentially empty lcov.info file (it contains a single line, TN:). I've since confirmed that this isn't unique to Android, I see the same effect on Linux (to reproduce, run the first two or three blocks of the above shell code, with $features and $this_repo replaced or set as needed, then run grcov). I'm not sure whether this is a bug in how I'm trying to generate the coverage file, or a bug in grcov, but any ideas would be appreciated.

Copy link

stale bot commented Apr 26, 2025

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Apr 26, 2025
@jtracey
Copy link
Contributor Author

jtracey commented Apr 29, 2025

This is being tracked in #6857 now. I don't think Android ever had working coverage, but in any case it's blocked on getting something working in the general case. If we want a separate issue to track, we can open one once we know what the goal is.

@jtracey jtracey closed this as completed Apr 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant