Get test coverage for software written in Rust using Kcov

2018-01-31 12:04:00 UTC

To get test coverage for your Rust code, you can use kcov.

It generates pretty HTML and JSON report files that include also coverage score. And, this one liner can extract percentage value from index.json, if you want.

Ohh, 4.5% .. !!! Anyway, if you run the test on GitLab CI, there is pattern matching on general settings, So we can just like this:

And then, use pattern something like this:

This seems that it usually works well. But I faced still some problems on CI.

Trouble Shooting

SIGSEGV

According to issue #212 (SimonKagstrom/kcov), you can pass --include-path to avoid this SIGSEGV error.

With this option, it worked fine also on my code. But, for now, in some project, it seems that it might not work, properly. I found pull request #55 (sagiegurari/cargo-make).

Cannot open linux-vdso.so.1

In local machine (Gentoo Linux, 64bit), kcov which is installed via Portage, works fine with my Rust code, but I faced this error on GitLab CI (Debian). This problem also discussed on issue #26 (SimonKagstrom/kcov).

I've installed kcov via apt-get (with cache configuration).

test:
          stage: test
          image: rust:latest
          ...
          variables:
            APT_CACHE_DIR: apt-cache
          before_script:
            # this lines install kcov
            - mkdir -pv $APT_CACHE_DIR && apt-get -qq update
            - apt-get -qq -o dir::cache::archives="$APT_CACHE_DIR" install -y kcov
          ...
          cache:
            untracked: true
            paths:
              - apt-cache
          ...
        
test:
          ...
          after_script:
            - kcov --include-path src target/coverage target/debug/<name>-*
            - cat target/coverage/index.json
          ...
        
: output on GitLab CI
        $ kcov --include-path src target/coverage target/debug/20min-*
        
        running 0 tests
        
        test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out
        
        Error: Cannot open linux-vdso.so.1

Hmm, it seems that kcov on Debian is a little bit old (11.1?). And it does not have even `--version` option. See links below (In 01.02.2018, 11.1 is still stable version.)

So, finally, I wrote a build script using newest version (version 34) with cache support on CI.

Build Script

This is a small bash script to build/install kcov into kcov directory on project root.

And .gitlab-ci.yml will be something like this ( In addition to dependencies listed in INSTALL.md of kcov, you need also cmake):

Set KCOV_DISCARD_CACHE as true, if you need to force re:install kcov. The kcov directory will be normally cached on GitLab CI!

I made also make cov target to check it on local, and a ci-runner script run CI docker container on local.

Check it out on my small project repo grauwoelfchen/20min ;)

References