Test Insights

Using CircleCI test Insights, you are able to analyze your test performance within the most recent executions and identify opportunities for test optimization. To access test insights, select the Tests tab on the Workflow Insights page.

The next sections go over what test insights are available.

The summary section displays test suite performance across your most recent 100 runs. You can view the following:

• Average test per run
• Number of all flaky tests detected
• Failure counts
• Slow run times in test suite

The most recent runs chart displays the most recent 100 executions of the test suite. You can hover over each bar to review the following:

• Test count
• Skipped tests
• Success rate of tests

Flaky tests are tests that fail non-deterministically; they pass and fail due to some state external to the repository. As a result, flaky tests introduce uncertainty to a project’s build state.

Test Insights detect flaky tests by identifying tests that failed and passed on the same commit in a 14-day window. Deterministic tests rely on the state of the repository and demonstrate the same behavior on re-runs. Additionally, these types of tests are labeled “FLAKY” throughout the CircleCI app making them easy to identify and fix.

You can view the 100 tests with the lowest success rates in their most recent pipeline executions. The table provides the following:

• Test name
• Associated job
• Run time
• Success rate

You can view the 100 tests with the longest run times in their most recent pipeline executions. The table provides the following:

• Test name
• Associated job
• Run time
• Success rate