How I can get coverage for cargo test?

Question

When I want to test C++ coverage, I can build my program with -fprofile-arcs -ftest-coverage, run all tests, and run gcov to get coverages.

However, when it comes to Rust, I get totally lost. What I want to do is run the following tests(on my Mac), and get coverage of all Rust codes in path components/raftstore

cargo test --package tests --test failpoints cases::test_normal
cargo test --package tests --test failpoints cases::test_bootstrap
cargo test --package tests --test failpoints cases::test_compact_log

From this post, it says firstly run cargo test --no-run, then run kcov. However, when I actually do that, kcov blocks forever.

Then I find something called cargo kcov, who provides --test. However when I run cargo kcov --test failpoints cases::test_normal like what I do in cargo test, I get error

error: cargo subcommand failure
note: cargo test exited with code exit status: 101
error: no test target named `failpoints`

I have tried many ways to figure this out, however, none of them works, so I wonder if I can get some help here.

I know there are other coverage tools like tarpaulin and grcov, I am currently trying those. It is also acceptable if there are neat solutions with those coverage tools. However, I still want to know what is wrong with kcov and cargo-kcov.

Answer 1

According to rustc docs, it's now possible to obtain instrumentation-based code coverage.

The following command generates coverage results. Note that it requires the Rust profiler runtime, which is included by default in nightly.

RUSTFLAGS="-C instrument-coverage" \
    cargo test --tests

These results may include mangled symbol names, which can be worked around with rustfilt:

cargo install rustfilt

After running tests with this setup there should be one or more .profraw files. If there are multiple, they can be merged with:

$ llvm-profdata merge -sparse default_*.profraw -o your_crate.profdata

Coverage information can then be shown with llvm-cov, for instance like described in this section (substitute the names of your crate and test binaries).

There might be issues if the LLVM versions of rustc and llvm-profdata resp. llvm-cov don't match, as noted here.

This answer is based on this section of the rustc book, which offers much more information on code coverage.

Answer 2

I like to use tarpaulin with the HTML output.

cargo tarpaulin --out Html

This should create an output file named tarpaulin-report.html. Open this file in the browser to view the code coverage report.

Answer 3

You can generate source-based coverage for your project by:

• Enable coverage profile in Cargo.toml:

  ...
  [build]
  profiler = true
  ...
  

• Install demangler

  cargo install rustfilt
  

• Add instrument coverage

  export RUSTFLAGS="-C instrument-coverage=all"
  

• Install dependencies openssl-devel, libssl-devel, llvm, and genhtml, based on your OS

• Install Mozilla's grcov and link the profiler with the rust compiler

  cargo install grcov
  rustup component add llvm-tools-preview
  

• Run tests

  cargo test --verbose
  

• Generate code coverage information

  grcov . -s . --binary-path ./target/debug/ -t lcov --branch --ignore-not-existing -o ./target/debug/
  

• Create HTML report from coverage data

  genhtml -o ./target/debug/coverage/ --show-details --highlight --ignore-errors source --legend ./target/debug/lcov
  

And finally view the HTML report at ./target/debug/coverage/index.html

Answer 4

The answers from Mori and from Saurabh describe in some detail how to configure rustc to provide instrumentation-based code coverage.

It is now possible to benefit from instrumentation-based coverage much more simply. cargo-llvm-cov can do this in a single command:

cargo llvm-cov

Note that as of 0.5.37, it does not have branch coverage (see https://github.com/taiki-e/cargo-llvm-cov/issues/8) or stable support for doc tests (see https://github.com/taiki-e/cargo-llvm-cov/issues/2).

Likewise, tarpaulin mentioned in the answer from Jim has been updated to use rustc's llvm instrumentation. By default, this is only used on macOS and Windows. On Linux it can be enabled with

cargo tarpaulin --engine llvm