Test splitting is one of the easiest ways to speed up builds on CircleCI. In particular, splitting tests by timing data can dramatically improve build times. This tutorial describes how test splitting with CirlceCI works under the hood. We’ll set up a React.js application with jest-junit to save test results so that you can split by timing data on subsequent runs.

Prerequisites

  • Basic knowledge of React.js
  • Node.js installed on your system
  • A CircleCI account
  • The Yarn package manager (this process can also be applied to npm)

What to expect with test splitting

CircleCI takes in a list of tests and splits those tests across the number of nodes defined by the parallelism key. Each node is its own separate container, so each one will need to spin up, check out the code, and perform any steps required to run the tests. The step that runs your test can make a difference in time between the steps that are run on each container.

Find out why CircleCI Rob Zuber thinks test splitting is an important part of intelligent CI/CD.

If it takes about 60 seconds to spin up the container, checkout the code, and install cached dependencies, you should expect that to be consistent across all parallel nodes. Each of these steps is identical across the containers. The test step is the only unique step we run on each container. This is why parallelism of 2 does not cut the time by precisely 1/2.

In many tests, parallelism dramatically reduces the time needed to perform lengthy steps. The CircleCI CLI disperses the tests so that the steps finish as close to evenly paced as possible. That said, there is no magic parallelism number. Each test suite is different, so we recommend experimenting to find the optimal number of nodes for you.

Test splitting with Yarn

To split tests, you give the CircleCI CLI a list of test files or classnames to split.

Like many other test suites, Jest doesn’t accept test files as a list of files or classnames. By default, Jest looks for files fitting its naming convention. That’s not a problem. We can provide that by globbing our test files in the way that Jest looks for them and piping them to the CLI as standard input.

            - run:
                name: Test application
                command: |
                    TEST=$(circleci tests glob "src/__tests__/*.js" | circleci tests split)
                    yarn test $TEST

This example looks for any .js file located in any __tests__ subdirectory. If your setup is different, globster.xyz is an extremely helpful tool to experiment with globbing patterns like brace expansion.

The CLI can split by name, filesize, or historical timing data (more about this later). For now, because nothing is defined, the CLI will fall back to splitting by name.

Splitting by timing data

For most projects, splitting tests by timing data is the fastest. CircleCI looks at historical timing data and dynamically splits tests across the nodes to have the test suite finish as uniformly as possible. To enable splitting by timing, set up your pipeline to collect and store the test results. We’ll be using jest-junit for this step.

There are a few things you need to add to our project to make this work. Start by installing it as a dev dependency:

yarn add --dev jest-junit

Next, update your test splitting step to split by timing data:

            - run:
                name: Test application
                command: |
                    TEST=$(circleci tests glob "src/__tests__/*.js" | circleci tests split --split-by=timings)
                    yarn test $TEST

Next, you need to make a couple of changes in the package.json file.

  • Add a reporters entry to the jest config
  • Add a jest-junit entry
  • Let jest-junit know to add a file attribute to the test results

CircleCI needs this information to be able to sift through the timing data. These additions will ensure that test results will be stored properly:

...
  "jest": {
    ...
    "reporters": [
      "default",
      "jest-junit"
    ],
    ...
  }
  ...
  "jest-junit": {
    "addFileAttribute": "true"
  },
...

Next, let CircleCI know that it needs to store this information using the store_test_results step. Each node’s results can be saved as an artifact related to the job using the store_artifacts step. Here is our final testing job:

    build-and-test:
        docker:
            - image: cimg/node:12.16
        parallelism: 5
        steps:
            - checkout
            - node/install-packages:
                pkg-manager: yarn      
            - run: mkdir ~/junit
            - run:
                name: Test application
                command: |
                    TEST=$(circleci tests glob "src/__tests__/*.js" | circleci tests split --split-by=timings)
                    yarn test $TEST
            - run:
                command: cp junit.xml ~/junit/
                when: always
            - store_test_results:
                path: ~/junit
            - store_artifacts:
                path: ~/junit

Confirming that everything works

To confirm that timing data is being used, check the logs for the testing step. Test logs

Conclusion

That’s it! We covered everything you need to know to run your test suites in parallel and store your timing data.

The full repository for the project used in this tutorial is available here.