Generating code coverage metrics
Code Coverage tells you how much of your application is tested.
CircleCI provides a number of different options for code coverage reporting, using built-in CircleCI features combined with open source libraries, or using partners.
See coverage in CircleCI
It’s straightforward to see simple coverage results from your build. Simply add a coverage library to your project, and configure it to write the results out to CircleCI’s artifacts directory. CircleCI will upload coverage results and make them visible as part of your build.
Adding and configuring a coverage library
In Ruby 1.9, you add the simplecov gem and configure your test suite to add code coverage.
In your Gemfile, add the
gem 'simplecov', :require => false, :group => :test
or in your test suite’s startup hooks, initialize
require 'simplecov' # save to CircleCI's artifacts directory if we're on CircleCI if ENV['CIRCLE_ARTIFACTS'] dir = File.join(ENV['CIRCLE_ARTIFACTS'], "coverage") SimpleCov.coverage_dir(dir) end SimpleCov.start
The simplecov README has more details.
In Scala 2 you can use the sbt-scoverage plugin.
Add the plugin to
addSbtPlugin("org.scoverage" % "sbt-scoverage" % "1.5.0")
For a simple project, enable coverage in
sbt coverage test:compile.
dependencies: override: - sbt coverage test:compile
Run tests with
sbt coverage test and produce a coverage report with
test: override: - sbt coverage test - sbt coverageReport post: - mkdir -p $CIRCLE_ARTIFACTS/scala-2.10 - mv target/scala-2.10/coverage-report $CIRCLE_ARTIFACTS/scala-2.10/coverage-report - mv target/scala-2.10/scoverage-report $CIRCLE_ARTIFACTS/scala-2.10/scoverage-report
See gslowikowski/sbt-simple-circleci-test for a complete example.
For multimodule projects, run tests against multiple Scala versions with
sbt coverage +test and aggregate reports with
sbt coverageAggregate. See gslowikowski/sbt-multimodule-circleci-test for a complete example.
Python, Node, Java, PHP, etc
We’re working on a guide for other languages. In the meantime, add your coverage library of choice. Options include:
- coverage for python
- node-coverage, blanket.js, jscoverage or istanbul for Node
- php-code-coverage or Atoum for PHP
gcovfor C or C++.
Configure your library to save results in the directory specified by the CIRCLE_ARTIFACTS environment variable. Alternatively, add a test.post command to copy your artifacts into the artifacts directory.
Seeing the results in the CircleCI UI
We will upload your coverage files, which will allow you view them from the “Artifacts” tab on the build page:
You can also get them via the CircleCI API.
And then of course, by viewing the generated HTML, you can see beautifully rendered HTML in the UI.
Using a Code Coverage service
You can easily send coverage results from CircleCI to a number of external code quality services:
If you’re a Codecov customer, integration with CircleCI can be as easy as
test: post: - bash <(curl -s https://codecov.io/bash)
No upload token is required for open source repositories.
Codecov’s Bash uploader detects all coverage reports from all languages and details for the CircleCI build to create a single, cohesive coverage report. Codecov will automatically handle the merging of coverage stats in parallel builds.
For more information find an example of your language here: https://github.com/codecov?query=example
Coveralls will automatically handle the merging of coverage stats in parallel builds.
- In the Code Climate UI, go to your repo and then navigate to Settings -> Test Coverage.
- Grab your test reporter token and add it to your CircleCI environment variables as
- Follow the Code Climate documentation for your specific language or framework.
Code Climate does not currently support CircleCI’s parallelization.
Code coverage is an area where our support is evolving. We’d love to have your feedback about how we could support it better. Let us know how you’re using it, how you’d like to use it, and what we could do better!