One of the easiest ways to speed up builds on CircleCI is with test splitting. Particularly, splitting tests by timing data. This tutorial details how test splitting with CirlceCI works under the hood. We’ll set up a React.js application with jest-junit to save test results so that you can split by timing data on subsequent runs.

Prerequisites

  • Basic knowledge of React.js
  • Node.js installed on your system
  • A CircleCI account
  • The Yarn package manager (this process can also be applied to npm)

What to expect with test splitting

The CircleCI test splitting mechanism takes in a list of tests and splits those tests across the number of nodes defined by the parallelism key. Each node is its own separate container, so each one will need to spin up, check out the code, and perform any steps required to run the tests. The difference in time between the steps run on each container is due to whichever step runs your tests. So if it takes ~60 seconds to spin up the container, checkout the code, and install cached dependencies, you should expect that to be consistent across all parallel nodes. Each of these steps is identical across the containers. The test step is the only unique step we run on each container. This is why parallelism of 2 does not cut the time by precisely 1/2!

Parallelism will drastically reduce the time needed to perform lengthy steps with many tests. The CircleCI CLI disperses the tests so that the steps will finish as close to evenly as possible. Although the magic under the hood will try to have everything finish at the same time, there is no magic parallelism number. Each test suite is different, so we recommend experimenting to see the optimal number of nodes for you.

Test splitting with Yarn

The CircleCI CLI is needed to split tests. For that to happen, it needs to receive a list of test files or classnames to split.

Like many other test suites, Jest doesn’t accept test files as a list of files or classnames. By default, Jest looks for files fitting its naming convention. That’s not a problem. We can provide that by globbing our test files in the way that Jest looks for them and piping them to the CLI as standard input.

            - run:
                name: Test application
                command: |
                    TEST=$(circleci tests glob **/__tests__/*.js | circleci tests split)
                    yarn test $TEST

This example looks for any .js file located in any __tests__ subdirectory. If your setup is different, globster.xyz is an extremely helpful tool to experiment with globbing patterns like brace expansion.

The CLI can split by name, filesize, or historical timing data (more on this below). For now, because nothing is defined, the CLI will fall back to splitting by name.

Splitting by timing data

For most, splitting tests by timing data is the fastest. CircleCI looks at historical timing data and dynamically splits tests across the nodes to have the test suite finish as uniformly as possible. To enable splitting by timing, we need to set up our pipeline to collect and store the test results. We’ll be using jest-junit for this piece. We have a few things that we need to add to our project to make this work. Start by installing it as a dev dependency:

yarn add --dev jest-junit

Next, our test splitting step should be updated to split by timing data:

            - run:
                name: Test application
                command: |
                    TEST=$(circleci tests glob **/__tests__/*.js | circleci tests split --split-by=timings)
                    yarn test $TEST

Next, we need to do a couple of things in the package.json file. Add a reporters entry to the jest config. Also add a jest-junit entry. We need to let jest-junit know to add a file attribute to the test results. CircleCI needs this to be able to sift through the timing data. These additions will ensure that test results will be stored:

...
  "jest": {
    ...
    "reporters": [
      "default",
      "jest-junit"
    ],
    ...
  }
  ...
  "jest-junit": {
    "addFileAttribute": "true"
  },
...

The next step is to let CircleCI know that it needs to store this information using the store_test_results step. Additionally, each node’s results can be saved as an artifact related to the job using the store_artifacts step. Here is our final testing job:

    build-and-test:
        docker:
            - image: cimg/node:12.16
        parallelism: 5
        steps:
            - checkout
            - node/install-packages:
                pkg-manager: yarn      
            - run: mkdir ~/junit
            - run:
                name: Test application
                command: |
                    TEST=$(circleci tests glob **/__tests__/*.js | circleci tests split --split-by=timings)
                    yarn test $TEST
            - run:
                command: cp junit.xml ~/junit/
                when: always
            - store_test_results:
                path: ~/junit
            - store_artifacts:
                path: ~/junit

Confirming that everything works

To confirm that timing data is being used, check the logs for the testing step. Test logs

Conclusion

That’s it! We covered everything you need to know to run your test suites in parallel and store your timing data.

The full repository for the project used in this tutorial is available here.