No results for

TutorialsLast Updated Jan 28, 20255 min read

A guide to test splitting

Ryan Pedersen

Solutions Engineer

2020-08-18-TestSplitting-Revamp-v1

Test splitting is one of the easiest ways to speed up builds on CircleCI. In particular, splitting testing by timing data can dramatically improve build times. This tutorial describes how test splitting with CirlceCI works under the hood. We’ll set up a React.js application with jest-junit to save test results so that you can split by timing data on subsequent runs.

Prerequisites

  • Basic knowledge of React.js
  • Node.js installed on your system
  • A CircleCI account
  • The Yarn package manager (this process can also be applied to npm)

What to expect with test splitting

CircleCI takes in a list of tests and splits those tests across the number of nodes defined by the parallelism key. Each node is its own separate container, so each one will need to spin up, check out the code, and perform any steps required to run the tests. The step that runs your test can make a difference in time between the steps that are run on each container.

Find out why CircleCI Rob Zuber thinks test splitting is an important part of intelligent CI/CD.

If it takes about 60 seconds to spin up the container, checkout the code, and install cached dependencies, you should expect that to be consistent across all parallel nodes. Each of these steps is identical across the containers. The test step is the only unique step we run on each container. This is why parallelism of 2 does not cut the time by precisely 1/2.

In many tests, parallelism dramatically reduces the time needed to perform lengthy steps. The CircleCI CLI disperses the tests so that the steps finish as close to evenly paced as possible. That said, there is no magic parallelism number. Each test suite is different, so we recommend experimenting to find the optimal number of nodes for you.

Test splitting with npm

To split tests, you give the CircleCI CLI a list of test files to split.

By default, Jest looks for test files based on a specific naming convention. This isn’t an issue, as we can capture these files using a glob pattern and pipe them to the CLI for test splitting.

- run:
    name: Test application
    command: |
      TEST=$(circleci tests glob "src/__tests__/*.js" | circleci tests split --split-by=timings)
      npm test $TEST
Copy to clipboard

This command looks for any .js file located in any __tests__ subdirectory. If your setup differs, you can use globster.xyz to experiment with glob patterns.

The CircleCI CLI can split by name, filesize, or historical timing data (more about this later). For now, because nothing is defined, the CLI will fall back to splitting by name.

Splitting by timing data

For most projects, splitting tests by timing data is the optimal choice. CircleCI analyses timing data from previous runs and splits tests across nodes to balance the runtime. To enable this, configure your CI/CD pipeline to collect and store test results using jest-junit.

Configuring Jest to use jest-junit

To set up jest-junit so it works both locally and in CircleCI, you’ll need to add configuration settings directly in the package.json file. This will enable Jest to generate JUnit test reports consistently, regardless of whether tests are run locally or in CircleCI.

Start by installing jestjunit as a dev dependency:

npm install --save-dev jest-junit
Copy to clipboard

Next, you need to make a couple of changes in the package.json file:

  • Jest Reporter Configuration: This configures Jest to use both the default reporter and jest-junit, ensuring the test results are output in a JUnit format.
  • jest-junit Settings: This configures jest-junit to include a file attribute in the test results, which is useful for CircleCI when analyzing timing data.

Add or modify the following in your package.json file:

{
  // ... other configuration in package.json
  "jest": {
    // Existing Jest configuration
    "reporters": [
      "default",
      [
        "jest-junit",
        {
          "outputDirectory": "./junit",
          "outputName": "junit.xml",
          "addFileAttribute": "true"
        }
      ]
    ]
  },
  "jest-junit": {
    "addFileAttribute": "true"
  }
}
Copy to clipboard

Here is what each settings does:

  • outputDirectory: Specifies the directory where the junit.xml file will be saved. This is set to ./junit, so the report will be stored in a folder named junit in the root of the project.
  • outputName: Specifies the name of the output file as junit.xml.
  • addFileAttribute: Adds a file attribute to the JUnit XML output, which helps for CircleCI parse and store test timings.

CircleCI Configuration

Next, let CircleCI know that it needs to store this information using the store_test_results step. Each node’s results can be saved as an artifact related to the job using the store_artifacts step.

Here is our final testing job:

version: 2.1

orbs:
  node: circleci/node@5.0.0

jobs:
  build-and-test:
    docker:
      - image: cimg/node:16.20
    parallelism: 5
    steps:
      - checkout
      - run:
          name: Install Packages with npm
          command: npm install
      - run: mkdir -p ~/junit # Create the junit directory
      - run:
          name: Run Tests with Split Timing
          command: |
            # Capture test files and split them by timing
            TEST_FILES=$(circleci tests glob "src/__tests__/*.js" | circleci tests split --split-by=timings)
            npm test $TEST_FILES
      - run:
          name: Copy JUnit Report
          command: cp ./junit/junit.xml ~/junit/
          when: always
      - store_test_results:
          path: ~/junit
      - store_artifacts:
          path: ~/junit

workflows:
  build-and-test:
    jobs:
      - build-and-test
Copy to clipboard

Here, we have a job that runs tests in parallel across five nodes. The store_test_results step saves the test results to the CircleCI server, and the store_artifacts step saves the JUnit report as an artifact.

Confirming that everything works

Commit the changes and push your project to GitHub. Log in to CircleCI and search for your project.

Search for project

Click on the “Set Up Project” button to start building your project.

CircleCI workflow

To confirm that timing data is being used, check the logs for the testing step.

Test logs

Conclusion

That’s it! We covered everything you need to know to run your test suites in parallel and store your timing data.

With this setup, we’ve ensured that jest-junit configurations are consistent across both local and CI environments. The configuration in package.json allows for seamless test splitting by timing data on CircleCI while keeping the local setup aligned. By leveraging CircleCI’s test-splitting capabilities and jest-junit, you can optimize your build and testing process, making it faster and more efficient.

The full repository for the project used in this tutorial is available here.

Copy to clipboard