Get test coverage for software written in Rust using Kcov
2018-01-31 12:04:00 UTCTo get test coverage for your Rust code, you can use kcov.
% cargo test % kcov --include-path ./src ./target/coverage target/debug/<name>-*
It generates pretty HTML and JSON report files that include also coverage score. And, this one liner can extract percentage value from index.json
, if you want.
% grep -oE 'covered":"([0-9]*\.[0-9]*|[0-9]*)"' target/coverage/index.json | \ grep -oE '[0-9]*\.[0-9]*|[0-9]*' 4.5
Ohh, 4.5%
.. !!! Anyway, if you run the test on GitLab CI, there is pattern matching on general settings, So we can just like this:
: in .gitlab-ci.yml % cat target/coverage/index.json
And then, use pattern something like this:
"covered":"(\d+(?:\.\d+)?)",This seems that it usually works well. But I faced still some problems on CI.
Trouble Shooting
SIGSEGV
According to issue #212 (SimonKagstrom/kcov), you can pass --include-path
to avoid this SIGSEGV error.
With this option, it worked fine also on my code. But, for now, in some project, it seems that it might not work, properly. I found pull request #55 (sagiegurari/cargo-make).
Cannot open linux-vdso.so.1
In local machine (Gentoo Linux, 64bit), kcov
which is installed via Portage, works fine with my Rust code, but I faced this error on GitLab CI (Debian). This problem also discussed on issue #26 (SimonKagstrom/kcov).
I've installed kcov via apt-get
(with cache configuration).
test: stage: test image: rust:latest ... variables: APT_CACHE_DIR: apt-cache before_script: # this lines install kcov - mkdir -pv $APT_CACHE_DIR && apt-get -qq update - apt-get -qq -o dir::cache::archives="$APT_CACHE_DIR" install -y kcov ... cache: untracked: true paths: - apt-cache ...
test: ... after_script: - kcov --include-path src target/coverage target/debug/<name>-* - cat target/coverage/index.json ...
: output on GitLab CI $ kcov --include-path src target/coverage target/debug/20min-* running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out Error: Cannot open linux-vdso.so.1
Hmm, it seems that kcov on Debian is a little bit old (11.1?). And it does not have even `--version` option. See links below (In 01.02.2018, 11.1 is still stable version.)
So, finally, I wrote a build script using newest version (version 34) with cache support on CI.
Build Script
This is a small bash script to build/install kcov into kcov directory on project root.
#!/bin/bash set -eu # NOTE: # if set KCOV_DISCARD_CACHE=true, then it will force installing kcov) renew="${KCOV_DISCARD_CACHE:-false}" kcov_dir="kcov" kcov_bin="${kcov_dir}/bin/kcov" kcov_url="https://github.com/SimonKagstrom/kcov/archive" kcov_ver="v34" if [[ -f "${kcov_bin}" && "${renew}" != "true" ]]; then echo "kcov already installed in ${kcov_bin}" else rm -fr $kcov_dir mkdir $kcov_dir cd $kcov_dir curl -sLO ${kcov_url}/${kcov_ver}.tar.gz mkdir $kcov_ver tar zxvf ${kcov_ver}.tar.gz -C $kcov_ver --strip-components=1 cd $kcov_ver mkdir build cmake \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_INSTALL_PREFIX=/ make make install DESTDIR=../ fi
And .gitlab-ci.yml
will be something like this ( In addition to dependencies listed in INSTALL.md of kcov, you need also cmake
):
test: stage: test image: rust:latest variables: KCOV_DISCARD_CACHE: "false" APT_CACHE_DIR: apt-cache before_script: - mkdir -pv $APT_CACHE_DIR && apt-get -qq update - apt-get -qq -o dir::cache::archives="$APT_CACHE_DIR" install -y binutils-dev libcurl4-openssl-dev zlib1g-dev libdw-dev libiberty-dev cmake - ./bin/build-kcov - rustc --version - cargo --version - ./kcov/bin/kcov --version script: - cargo test after_script: - ./kcov/bin/kcov --include-path src target/coverage target/debug/20min-* - cat target/coverage/index.json cache: untracked: true paths: - apt-cache - kcov except: - tags
Set KCOVDISCARDCACHE
as true
, if you need to force re:install kcov. The kcov directory will be normally cached on GitLab CI!
I made also make cov target to check it on local, and a ci-runner script run CI docker container on local.
Check it out on my small project repo grauwoelfchen/20min ;)