Enabling Docker Layer Caching

If your application is distributed as a Docker image, the image consists of layers that generally change more frequently toward the bottom of the Dockerfile. This is because any lines that change in a Dockerfile invalidate the cache of that line and every line after it. The frequently changing layers are referred to as the top layers of the image after it is compiled.

Consider reusing the unchanged layers to significantly reduce image build times. By default, the Remote Docker Environment doesn’t provide layer caching, but you can enable this feature with a special option:

- setup_remote_docker:
    reusable: true    # default - false
    exclusive: true   # default - true

reusable

When reusable is set to true, CircleCI will try to reuse your Remote Docker Environment. That is, every layer you built in a previous job will be accessible in the remote environment and CircleCI will attempt to reuse the previous environment when it is possible. However, in some cases your job may run in a clean environment, even if the configuration specifies reusable: true.

If you run many parallel jobs for the same project, and each job requests a reusable environment, all of them will be provided with a Remote Docker Environment. However, not all of the jobs will have cached layers, although this behavior is subject to change in a future update.

exclusive

The second option (exclusive) indicates whether you want to allow parallel jobs to run simultaneously in the same Remote Docker Environment. Jobs for which exclusive: is set to true are guaranteed to have an exclusive Remote Docker Environment that other jobs cannot access.

If you set exclusive: to false, the project’s parallel jobs will be able to share the same Remote Docker Environment. This method lets you reduce the chances of receiving a Remote Docker Environment without a cache. If you choose this option, ensure that your project can handle concurrent operations in Docker Engine.

For example, if you build images with mutable tags like latest, then a shared Docker environment could have undesireable results. When exclusive: is set to false, CircleCI only allows two parallel jobs to use the same Docker environment.