Test splitting is one of the easiest ways to speed up builds on CircleCI. In particular, splitting testing by timing data can dramatically improve build times. This tutorial describes how test splitting with CirlceCI works under the hood. We’ll set up a React.js application with jest-junit to save test results so that you can split by timing data on subsequent runs.
Prerequisites
- Basic knowledge of React.js
- Node.js installed on your system
- A CircleCI account
- The Yarn package manager (this process can also be applied to npm)
What to expect with test splitting
CircleCI takes in a list of tests and splits those tests across the number of nodes defined by the parallelism key. Each node is its own separate container, so each one will need to spin up, check out the code, and perform any steps required to run the tests. The step that runs your test can make a difference in time between the steps that are run on each container.
Find out why CircleCI Rob Zuber thinks test splitting is an important part of intelligent CI/CD.
If it takes about 60 seconds to spin up the container, checkout the code, and install cached dependencies, you should expect that to be consistent across all parallel nodes. Each of these steps is identical across the containers. The test step is the only unique step we run on each container. This is why parallelism of 2 does not cut the time by precisely 1/2.
In many tests, parallelism dramatically reduces the time needed to perform lengthy steps. The CircleCI CLI disperses the tests so that the steps finish as close to evenly paced as possible. That said, there is no magic parallelism number. Each test suite is different, so we recommend experimenting to find the optimal number of nodes for you.
Test splitting with npm
To split tests, you give the CircleCI CLI a list of test files to split.
By default, Jest looks for test files based on a specific naming convention. This isn’t an issue, as we can capture these files using a glob pattern and pipe them to the CLI for test splitting.
- run:
name: Test application
command: |
TEST=$(circleci tests glob "src/__tests__/*.js" | circleci tests split --split-by=timings)
npm test $TEST
This command looks for any .js
file located in any __tests__
subdirectory. If your setup differs, you can use globster.xyz to experiment with glob patterns.
The CircleCI CLI can split by name, filesize, or historical timing data (more about this later). For now, because nothing is defined, the CLI will fall back to splitting by name.
Splitting by timing data
For most projects, splitting tests by timing data is the optimal choice. CircleCI analyses timing data from previous runs and splits tests across nodes to balance the runtime. To enable this, configure your CI/CD pipeline to collect and store test results using jest-junit
.
Configuring Jest to use jest-junit
To set up jest-junit
so it works both locally and in CircleCI, you’ll need to add configuration settings directly in the package.json
file. This will enable Jest to generate JUnit test reports consistently, regardless of whether tests are run locally or in CircleCI.
Start by installing jestjunit
as a dev dependency:
npm install --save-dev jest-junit
Next, you need to make a couple of changes in the package.json
file:
- Jest Reporter Configuration: This configures Jest to use both the default reporter and jest-junit, ensuring the test results are output in a JUnit format.
- jest-junit Settings: This configures jest-junit to include a file attribute in the test results, which is useful for CircleCI when analyzing timing data.
Add or modify the following in your package.json
file:
{
// ... other configuration in package.json
"jest": {
// Existing Jest configuration
"reporters": [
"default",
[
"jest-junit",
{
"outputDirectory": "./junit",
"outputName": "junit.xml",
"addFileAttribute": "true"
}
]
]
},
"jest-junit": {
"addFileAttribute": "true"
}
}
Here is what each settings does:
outputDirectory
: Specifies the directory where thejunit.xml
file will be saved. This is set to./junit
, so the report will be stored in a folder namedjunit
in the root of the project.outputName
: Specifies the name of the output file asjunit.xml
.addFileAttribute
: Adds a file attribute to the JUnit XML output, which helps for CircleCI parse and store test timings.
CircleCI Configuration
Next, let CircleCI know that it needs to store this information using the store_test_results
step. Each node’s results can be saved as an artifact related to the job using the store_artifacts
step.
Here is our final testing job:
version: 2.1
orbs:
node: circleci/node@5.0.0
jobs:
build-and-test:
docker:
- image: cimg/node:16.20
parallelism: 5
steps:
- checkout
- run:
name: Install Packages with npm
command: npm install
- run: mkdir -p ~/junit # Create the junit directory
- run:
name: Run Tests with Split Timing
command: |
# Capture test files and split them by timing
TEST_FILES=$(circleci tests glob "src/__tests__/*.js" | circleci tests split --split-by=timings)
npm test $TEST_FILES
- run:
name: Copy JUnit Report
command: cp ./junit/junit.xml ~/junit/
when: always
- store_test_results:
path: ~/junit
- store_artifacts:
path: ~/junit
workflows:
build-and-test:
jobs:
- build-and-test
Here, we have a job that runs tests in parallel across five nodes. The store_test_results
step saves the test results to the CircleCI server, and the store_artifacts
step saves the JUnit report as an artifact.
Confirming that everything works
Commit the changes and push your project to GitHub. Log in to CircleCI and search for your project.
Click on the “Set Up Project” button to start building your project.
To confirm that timing data is being used, check the logs for the testing step.
Conclusion
That’s it! We covered everything you need to know to run your test suites in parallel and store your timing data.
With this setup, we’ve ensured that jest-junit
configurations are consistent across both local and CI environments. The configuration in package.json
allows for seamless test splitting by timing data on CircleCI while keeping the local setup aligned. By leveraging CircleCI’s test-splitting capabilities and jest-junit
, you can optimize your build and testing process, making it faster and more efficient.
The full repository for the project used in this tutorial is available here.