Optimizations Overview

1 week ago • 3 min read

Cloud Server v3.x Server v2.x

This document provides an overview of ways to optimize your CircleCI configuration. Each optimization method will be described briefly, and present possible use cases for speeding up your jobs.

• Custom storage controls
• Docker image choice
• Docker layer caching
• Caching dependencies
• Workflows
• Workspaces
• Parallelism
• Resource class

Custom storage controls

The CircleCI web app provides controls to customize the storage retention period of workspaces, caches, and artifacts. You can find these settings by navigating to Plan > Usage Controls. By default, the storage period is 30 days for artifacts, and 15 days for caches and workspaces. These are also the maximum retention periods for storage. The maximum storage period is 30 days for artifacts, and 15 days for caches and workspaces.

See the Persisting Data page for more information on custom storage settings.

Docker image choice

Choosing the right docker image for your project can have huge impact on build time. For example, choosing a basic language image means dependencies and tools need to be downloaded each time your pipeline is run, whereas, if you choose or build an image that has these dependencies and tools already installed, this time will be saved for each build run. When configuring your projects and specifying images, consider the following options:

• CircleCI provides a range of convenience images, typically based on official Docker images, but with a range of useful language tools pre-installed.
• You can create your own images, maximizing specificity for your projects. To help with this we provide both a Docker image build wizard, and guidance for building images manually.

Docker layer caching

Docker layer caching is a feature that can help to reduce the build time of a Docker image in your build. DLC is useful if you find yourself frequently building Docker images as a regular part of your CI/CD process.

DLC is similar to caching dependencies, in that it saves the image layers that you build within your job, making them available on subsequent builds.

• Read more on the Docker Layer Caching page.

Caching dependencies

Caching should be one of the first things you consider when trying to optimize your jobs. If a job fetches data at any point, it is likely that you can make use of caching. A common example is the use of a package/dependency manager. If your project uses Yarn, Bundler, or Pip, for example, the dependencies downloaded during a job can be cached for later use rather than being re-downloaded on every build.

• Read more on the Caching Dependencies page.

Workflows

Workflows provide a means to define a collection of jobs and their run order. If at any point in your configuration you see a step where two jobs could happily run independent of one another, workflows may be helpful. Workflows also provide several other features to augment and improve your CI/CD configuration. Read more about workflows on the Workflow page.

• You can view examples of workflows in the CircleCI demo workflows repo.

Workspaces

Workspaces are used to pass along data that is unique to a run and is needed for downstream jobs. So, if you are using workflows, a job run earlier in your build might fetch data and then make it available later for jobs that run later in a build.

To persist data from a job and make it available to downstream jobs via the attach_workspace key, configure the job to use the persist_to_workspace key. Files and directories named in the paths: property of persist_to_workspace will be uploaded to the workflow’s temporary workspace relative to the directory specified with the root key. The files and directories are then uploaded and made available for subsequent jobs (and re-runs of the workflow) to use.

• Read more on the Workspaces page.

Parallelism

If your project has a large test suite, you can configure your build to use parallelism together with either CircleCI’s test splitting functionality, or a third party application or library to split your tests across multiple machines. CircleCI supports automatic test allocation across machines on a file-basis, however, you can also manually customize how tests are allocated.

• Read more about splitting tests on the Parallelism page.

Resource class

Note: If you are on a container-based plan, you will need to open a support ticket to enable this feature on your account. Resource class options for self-hosted installations are set by system administrators.

Using resource_class, it is possible to configure CPU and RAM resources for each job. For Cloud, see this table for a list of available classes, and for self-hosted installations contact your system administrator for a list.

Please note, if a resource_class is not explicitly declared, CircleCI will try to find the best default resource class for your organization.

• See the resource_class section of the Configuration Reference for more information.

See also

• Persisting Data
• For a complete list of customizations, view the Configuration Reference page.
• For information about how Yarn can potentially speed up builds and reduce errors, view the Using Yarn page.
• Coinbase published an article titled Continuous Integration at Coinbase: How we optimized CircleCI for speed and cut our build times by 75%.

---

Help make this document better

This guide, as well as the rest of our docs, are open source and available on GitHub. We welcome your contributions.

• Suggest an edit to this page (please read the contributing guide first).
• To report a problem in the documentation, or to submit feedback and comments, please open an issue on GitHub.
• CircleCI is always seeking ways to improve your experience with our platform. If you would like to share feedback, please join our research community.

Need support?

Our support engineers are available to help with service issues, billing, or account related questions, and can help troubleshoot build configurations. Contact our support engineers by opening a ticket.

You can also visit our support site to find support articles, community forums, and training resources.

---

CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.