Documentation structure for LLMs (llms.txt)
CircleCI Logo

Auto rerun failed tests

2 days ago · 7 min read
Cloud

Auto rerun failed tests automatically detects and retries test failures during your test run. Compared with manually Rerunning Failed Tests from a workflow, auto rerun failed tests eliminates the need to:

  • Manually identify which jobs failed.

  • Manually select Rerun workflow from failed in the CircleCI web app.

Test impact analysis, dynamic test splitting, dynamic parallelism, and auto rerun failed tests have a credit charge above your plan’s allowance. See Credit Charge.

Is my project a good fit for auto rerun failed tests?

This section outlines several project features that indicate your project is a good candidate for auto rerun failed tests:

Your test runner produces JUnit test results

Auto rerun failed tests identifies which test atoms to rerun from JUnit XML output. Your test runner must be configured to emit JUnit XML results for the feature to work.

Your test suite has flaky tests

If your test suite has flaky tests, auto rerun can quickly recover from intermittent failures without rerunning your entire test suite.

How it works

All test atoms are run as normal with the discover and run commands.

When auto rerun failed tests is enabled:

  • Any test atoms that failed in the JUnit XML output are rerun.

  • The process repeats until all test atoms pass, or the maximum number of rerun attempts is reached.

  • If all test atoms pass, the errors are suppressed and the job succeeds.

  • If the maximum number of reruns is reached and there are still test failures, the job fails.

flowchart LR subgraph Run direction TB R1a[A pass] --- R1b[B fail] --- R1c[C pass] --- R1d[D fail] --- R1e[E fail] end subgraph Rerun-1[Rerun 1] direction TB R2b[B pass] --- R2d[D fail] --- R2e[E pass] end subgraph Rerun-2[Rerun 2] direction TB R3d[D pass] end Run --> Rerun-1 --> Rerun-2 --> Done[All passed] style R1a fill:#00DB74,color:#fff style R1c fill:#00DB74,color:#fff style R1b fill:#FC5656,color:#fff style R1d fill:#FC5656,color:#fff style R1e fill:#FC5656,color:#fff style R2b fill:#00DB74,color:#fff style R2e fill:#00DB74,color:#fff style R2d fill:#FC5656,color:#fff style R3d fill:#00DB74,color:#fff style Done fill:#00DB74,color:#fff linkStyle 0 stroke:none linkStyle 1 stroke:none linkStyle 2 stroke:none linkStyle 3 stroke:none linkStyle 4 stroke:none linkStyle 5 stroke:none

Prerequisites

Before enabling auto rerun failed tests, ensure you have completed the Getting Started With Smarter Testing guide and have:

  • Configured your .circleci/test-suites.yml with discover and run commands.

  • Verified your tests run successfully with the testsuite command.

  • Correctly configured JUnit XML output in .circleci/test-suites.yml.

You can use the CircleCI Agent Skills to get Smarter Testing set up. Install the CircleCI Public Skills plugin to your AI coding agent and ask your agent to get set up for Smarter Testing.
This page covers max-auto-rerun in .circleci/test-suites.yml, which retries individual failed test atoms as part of Smarter Testing. If you want to retry entire steps or workflows, see Automatic Reruns, which uses the separate max_auto_reruns key in .circleci/config.yml.

1. Enable auto rerun failed tests in your .circleci/test-suites.yml file

Add the max-auto-rerun option to your .circleci/test-suites.yml:

---
name: ci tests
# ...
options:
  max-auto-rerun: 3
Options Field Default Description

max-auto-rerun

0

The maximum number of times test atoms are rerun if they fail. Accepts a value in the range 0 to 10 inclusive.

When auto-rerun-duration is also set, test atoms rerun until one of the two conditions is reached.

auto-rerun-duration

null

The duration or percentage of total runtime to spend rerunning failed test atoms, for example, 5m, 1h, 20%.

When max-auto-rerun is also set, test atoms rerun until one of the two conditions is reached.

2. Run

Use the doctor command to validate that your test-suites.yml is set up correctly. The CLI runs additional checks when auto rerun is enabled. If any results look incorrect, an action item is provided to resolve it.

$ circleci testsuite doctor "ci tests"

Follow the steps until all checks pass.

Run the circleci testsuite CLI tooling from the directory where your tests are located. That directory is typically your repository root, but for monorepos it can be the root of a subpackage (for example, cd service-1 && circleci testsuite run "ci tests").

The testsuite will look for the .circleci/test-suites.yml configuration file in the repository root or the subpackage where the CLI is run. All commands (discover, run, analysis) execute relative to where you run the CLI, so avoid using cd or --directory flags within your commands.

To verify auto rerun locally, modify a test to cause it to fail, then run the test suite:

$ circleci testsuite run "ci tests"

Commit .circleci/test-suites.yml to your feature branch and push to your VCS.

Verify auto rerun failed tests is working as expected in CI by checking the Tests tab in the CircleCI web app.

Next steps