cmd+/

Set parallelism dynamically

Cloud

Dynamic parallelism sets the parallelism of a test job when your pipeline runs, instead of using a fixed value in your configuration. An upstream job lists the test atoms that the test job will run. CircleCI then sets the test job’s parallelism from the test timings of those test atoms, within a minimum and maximum that you choose.

You can set parallelism dynamically in two ways:

Scale with the selected tests. CircleCI scales your maximum parallelism by the share of the test suite that the test job will run. Use this with Test Impact Analysis, so that the test job uses fewer parallel nodes when fewer test atoms are selected. See Scale with the selected tests.
Target a duration for each node. You set a target duration, and CircleCI sets enough parallel nodes to run for about that long. Use this with any test splitting, static or dynamic, to keep a test job near a desired duration as your test suite grows or shrinks, without adjusting parallelism by hand. See Target a duration for each node.

Test impact analysis, dynamic test splitting, dynamic parallelism, and auto rerun failed tests have a credit charge above your plan’s allowance. See Credit Charge.

Dynamic parallelism is experimental. The experimental.dynamic-parallelism configuration will be replaced in a future release. You will be required to update your configuration when the syntax changes.

Prerequisites

Before you set parallelism dynamically, ensure you have:

Completed the Getting Started With Smarter Testing guide.
Test timings for your test suite. Smarter Testing generates timings whenever it runs the full test suite. To generate them, merge to your default branch, or run the test suite once with the --run-tests=all flag.
To scale with the selected tests: set up Test Impact Analysis, which has a credit charge above your plan’s allowance. Without test impact analysis, list-tests selects the full test suite, and the test job always uses max parallel nodes.

How it works

An upstream job runs circleci testsuite list-tests to select the test atoms that the test job will run, without running them. list-tests applies the same selection rules as circleci testsuite run, so the selection matches what the test job runs. CircleCI then sets the test job’s parallelism from the test timings of the selected test atoms, within the min and max that you set.

Scale with the selected tests

With test impact analysis, test selection often runs only a small part of your test suite on feature branches. Dynamic parallelism scales the test job’s parallelism with the selected test atoms, so you do not run more parallel nodes than the selected test atoms need. On your default branch, where the full test suite runs, the test job uses max parallel nodes.

Target a duration for each node

With a fixed parallelism, a test job gets slower as you add tests, and you must keep adjusting the value to hold the job near a desired duration. Set a target instead, and CircleCI sets enough parallel nodes for each node to run the selected test atoms in about that time. Set max to cap the number of parallel nodes the test job can use.

A target works with both Static Test Splitting and Dynamic Test Splitting, and does not require test impact analysis. If you also use test impact analysis, CircleCI sets the parallelism from the runtime of the selected test atoms only.

The target is compared against test runtimes from your JUnit test results, which are not always close to the wall clock time of each node. For example, JUnit runtimes do not always include test runner startup or setup steps. With dynamic test splitting, CircleCI accounts for this difference. With static test splitting, you may need to tune the target until the test job’s nodes run for your desired wall clock time.

Set up dynamic parallelism

Add a job that runs circleci testsuite list-tests with a unique key. The key connects this job to the dynamic parallelism configuration for your test job.
```
jobs:
  list-tests:
    executor: my-executor
    steps:
      # Check out the code and install the dependencies the discover command needs.
      - setup
      # List the test atoms that would be selected, without running them.
      - run: circleci testsuite list-tests "ci tests" --key=ci-tests-parallelism-key
```
The list-tests job runs your discover command, so it needs a checkout and any dependencies that command relies on.

If your test job passes the --run-tests or --analyze-tests flags, pass the same flags to list-tests so the selection matches. See Overriding Tests Selected for Run and Analysis.

Run your test job after the list-tests job. Set the test job’s parallelism to the maximum a full test suite needs. If CircleCI cannot set the parallelism dynamically, the test job uses this value.

  test:
    executor: my-executor
    parallelism: 20
    steps:
      - setup
      - run: circleci testsuite run "ci tests"
      - store_test_results:
          path: test-reports

workflows:
  test-workflow:
    jobs:
      - list-tests
      - test:
          requires:
            - list-tests

Add an experimental.dynamic-parallelism stanza with the name of the test job, the key you passed to list-tests, and the minimum and maximum parallelism:
```
experimental:
  dynamic-parallelism:
    test:
      key: ci-tests-parallelism-key
      min: 1
      max: 20
```
Optional: to target a duration for each node, add target to the stanza:
```
      target: 2m
```
Set target to a duration between 8s and 5h, written as a number followed by a time unit, for example 5m or 1m30s. Valid time units are s, m, and h. If the target is outside this range or invalid, the test job uses the parallelism value in its configuration.
Push your changes and open the test job in the CircleCI web app. On the Steps tab, Parallel runs shows how many parallel nodes ran. Confirm that the number is between your min and max.

Example configuration

CircleCI configuration that sets parallelism to run each node for about 2 minutes

# .circleci/config.yml
version: 2.1

jobs:
  list-tests:
    executor: my-executor
    steps:
      # Check out the code and install the dependencies the discover command needs.
      - setup
      # List the test atoms that would be selected, without running them.
      - run: circleci testsuite list-tests "ci tests" --key=ci-tests-parallelism-key

  test:
    executor: my-executor
    # Set job parallelism to the maximum parallelism for a full suite to run.
    # Any failures will fall back to this value.
    # This is optional but recommended.
    parallelism: 20
    steps:
      - setup
      - run: circleci testsuite run "ci tests"
      - store_test_results:
          path: test-reports

workflows:
  test-workflow:
    jobs:
      - list-tests
      - test:
          requires:
            - list-tests

experimental:
  dynamic-parallelism:
    # The name of the job to set parallelism for.
    test:
      # Must match the key passed to list-tests.
      key: ci-tests-parallelism-key
      min: 1
      max: 20
      # Optional. The desired duration for each parallel node.
      # Remove target to scale with the selected tests.
      target: 2m

Next steps

Use Dynamic Test Splitting to evenly split the selected test atoms across the parallel nodes.
Advanced Configuration for test impact analysis.

[path]

[title]