Docker layer caching overview

Use Docker layer caching (DLC) to reduce Docker image build times on CircleCI. DLC is available on all CircleCI plans.

Docker layer caching (DLC) is beneficial if building Docker images is a regular part of your CI/CD process. DLC saves Docker image layers created within your jobs, and caches them to be reused during future builds.

DLC caches the individual layers of any Docker images built during your CircleCI jobs, and then reuses unchanged image layers on subsequent job runs, rather than rebuilding the entire image every time. In short, the less your Dockerfiles change from commit to commit, the less time your image-building jobs will take to run.

DLC can be used with both the machine executor and in a remote Docker environment (setup_remote_docker).

DLC is charged at 200 credits per job run. For more information about DLC charges, see the FAQ.

To use DLC with the Docker execution environment, you will need to configure your job to run in a Remote Docker Environment. To do this, add docker_layer_caching: true under the setup_remote_docker key in your .circleci/config.yml file:

version: 2.1

jobs:
 build:
    docker:
      - image: cimg/base:2023.04

    steps:
      - checkout
      - setup_remote_docker:
          docker_layer_caching: true

DLC can be used when building Docker images using the machine executor. Use DLC with the machine executor by adding docker_layer_caching: true below your machine key:

version: 2.1

jobs:
 build:
    machine:
      image: ubuntu-2004:202104-01  # any available image
      docker_layer_caching: true    # default - false

    steps:
      - checkout

DLC is only useful when creating your own Docker image with docker build, docker compose, or similar Docker commands. It does not decrease the wall clock time that all builds take to spin up the initial environment.

version: 2.1

orbs:
  browser-tools: circleci/browser-tools@1.2.3

jobs:
 build:
    docker:
      - image: cimg/node:17.2-browsers

    steps:
      - checkout
      - setup_remote_docker:
          docker_layer_caching: true # DLC will explicitly cache layers here and try to avoid rebuilding.
      - run: docker build .

When using buildx builder instances with DLC, it is important to name your builders. Doing so ensures CircleCI can detect and preserve the Docker volumes for subsequent job runs. If you do not name your builders, each time the job is run they will have different, randomly generated names, and the resulting volumes will be automatically cleaned up.

If you store Docker build artifacts in a Docker volume, managed by the buildkit inside buildx builder instances, the DLC feature cannot maintain these artifacts, but they can still be supported. DLC is not able to prune these images/build cache, but buildx builders do have some in-built pruning. For more information, see the Docker docs.

DLC caches your Docker image layers within the container/virtual machine used to run your job.

If, for example, the first run of your job takes over two minutes to build a Docker image, and nothing changes in the Dockerfile before the second run, the Dockerfile build steps happen instantly, in zero seconds.

When none of the layers in the image change between job runs, DLC pulls the layers from the cache and reuses those instead of rebuilding the entire image.

If part of the Dockerfile changes (which changes part of the image), a subsequent run of the exact same job with the modified Dockerfile may still finish faster than rebuilding the entire image. This is because the cache can still be used for the first few steps that did not change in the Dockerfile. The steps that follow the change must be rerun because the Dockerfile change invalidates the cache for those layers.

If you change something in your Dockerfile, all of the later steps (from the point that the change was made) are invalidated and the layers have to be rebuilt. When some of the steps remain the same (the steps before the one you removed), those steps can be reused. So, it is still faster than rebuilding the entire image.

There is a “DLC set-up” step at the beginning of each job that uses DLC. You are not charged for this step. At the end of each job that uses DLC, the cache upload is done asynchronously, and does not prevent the workflow from continuing to progress. This means that jobs within the same workflow are unlikely to access a cache uploaded from an upstream job. You are not charged for this “DLC teardown” step.

DLC operates in the same way for jobs that use parallelism. If a machine job using DLC is configured with parallelism: 2, two jobs run in parallel. Each virtual machine in this case will have a separate DLC cache, and whichever is saved last will be used for the next build.

DLC was previously enabled via the reusable: true key. The reusable key has been deprecated in favor of the docker_layer_caching key.

In addition, the exclusive: true option is deprecated and all remote Docker VMs are now treated as exclusive. This means that when using DLC, jobs are guaranteed to have an exclusive remote Docker environment that other jobs cannot access.