NAV

Configuration Reference

This document is a reference for .circleci/config.yml which describes jobs and steps to build/test/deploy your project. The presence of this file indicates that you want to use the 2.0 infrastructure. This allows you to test 2.0 builds on a separate branch, leaving any existing configuration in the old circle.yml style unaffected and running on the CircleCI 1.0 infrastructure in branches that do not contain .circleci/config.yml.

You can see a complete config.yml in our full example.


Table of Contents


version

Key Required Type Description
version Y String Should currently be 2

The version field is intended to be used in order to issue warnings for deprecation or breaking changes.

jobs

Each job is an item in the jobs list. build is the default job and is required in every config.yml. This is the only job that will be automatically picked up and run by CircleCI.

Each job consists of the job’s name as a key and a map as a value. A name should be unique within a current jobs list. The value map has the following attributes:

Key Required Type Description
docker Y (1) List Options for docker executor
machine Y (1) Map Options for machine executor
steps Y List A list of steps to be performed
working_directory Y String What directory to run the steps in. (previously called workDir).
parallelism N Integer Number of parallel instances of this job to run (default: 1)
environment N Map A map of environment variable names and valuables (NOTE: these will override any environment variables you set in the CircleCI web interface).
branches N Map A map defining rules for whitelisting/blacklisting execution of specific branches (default: all whitelisted)

(1) exactly one of them should be specified. It is an error to set more than one.

If parallelism is set to N > 1, then N independent executors will be set up and each will run the steps of that job in parallel. Certain parallelism-aware steps can opt out of the parallelism and only run on a single executor (for example deploy step). Learn more about parallel jobs.

working_directory will be created automatically if it doesn’t exist.

Example:

jobs:
  build:
    docker:
      - buildpack-deps:trusty
    environment:
      - FOO: "bar"
    parallelism: 3
    working_directory: ~/my-app
    branches:
      only:
        - master
        - /rc-.*/
    steps:
      - run: make test
      - run: make

docker | machine (executor)

An “executor” is roughly “a place where steps occur”. CircleCI 2.0 can build the necessary environment by launching as many docker containers as needed at once, or it can use a full virtual machine. Learn more about different executors.

docker

Configured by docker key which takes a list of maps:

Key Required Type Description
image Y String The name of a custom docker image to use
entrypoint N String or List The command used as executable when launching the container
command N String or List The command used as pid 1 (or args for entrypoint) when launching the container
user N String Which user to run the command as
environment N Map A map of environment variable names and values

entrypoint overrides default entrypoint from Dockerfile.

command will be used as arguments to image entrypoint (if specified in Dockerfile) or as executable (if no entrypoint is provided here or in the Dockerfile).

For primary container (listed first in the list) if no command is specified then command and image entrypoint will be ignored, to avoid errors caused by the entrypoint executable consuming significant resources or exiting preliminary. At this time all steps run in the primary container only.

The environment settings apply to all commands run in this executor, not just the initial command. The environment here has higher precedence over setting it in the job map above.

You can specify image versions using tags or digest. You can use any public images from any public Docker registry (defaults to Docker Hub). Learn more about specifying images.

Example:

jobs:
  build:
    docker:
      - image: buildpack-deps:trusty # primary container
        environment:
          ENV: CI

      - image: mongo:2.6.8
        command: [--smallfiles]

      - image: postgres:9.4.1
        environment:
          POSTGRES_USER: root

      - image: redis@sha256:54057dd7e125ca41afe526a877e8bd35ec2cdd33b9217e022ed37bdcf7d09673

machine

The usage of the machine executor is configured by using the machine key, which takes a map:

Key Required Type Description
enabled Y Boolean This must be true in order to enable the machine executor.

As a shorthand, you can set the machine key to true.

Example:

jobs:
  build:
    machine:
      enabled: true

# or just

jobs:
  build:
    machine: true

branches

Defines rules for whitelisting/blacklisting execution of some branches. Takes a map:

Key Required Type Description
only N List List of branches that only will be executed
ignore N List List of branches to ignore

Both only and ignore lists can have full names and regular expressions. For example:

branches:
  only:
    - master
    - /rc-.*/

In this case only “master” branch and branches matching regex “rc-.*” will be executed.

branches:
  ignore:
    - develop
    - /feature-.*/

In this example all the branches will be executed except “develop” and branches matching regex “feature-.*”.

If both ignore and only are present in config, only ignore will be taken into account.

A job that was not executed due to configured rules will show up in the list of jobs in UI, but will be marked as skipped.

steps

The steps setting in a job should be a list of single key/value pairs, the key of which indicates the step type. The value may be either a configuration map or a string (depending on what that type of step requires). For example, using a map:

jobs:
  build:
    working_directory: ~/canary-python
    environment:
      - FOO: "bar"
    steps:
      - run:
          name: Running tests
          command: make test

Here run is a step type. The name attribute is used by the UI for display purposes. The command attribute is specific for run step and defines command to execute.

Some steps may implement a shorthand semantic. For example, run may be also be called like this:

jobs:
  build:
    steps:
      - run: make test

In its short form, the run step allows us to directly specify which command to execute as a string value. In this case step itself provides default suitable values for other attributes (name here will have the same value as command, for example).

Another shorthand, which is possible for some steps, is to simply use the step name as a string instead of a key/value pair:

jobs:
  build:
    steps:
      - checkout

In this case, the checkout step will checkout project source code into the job’s working_directory.

In general all steps can be describe as:

Key Required Type Description
<step_type> Y Map or String A configuration map for the step or some string whose semantics are defined by the step.

Each built-in step is described in detail below.

run

Used for invoking all command-line programs, taking either a map of configuration values, or, when called in its short-form, a string that will be used as both the command and name. Run commands are executed using non-login shells by default, so you must explicitly source any dotfiles as part of the command.

Key Required Type Description
command Y String Command to run via the shell
name N String Title of the step to be shown in the CircleCI UI (default: full command)
shell N String Shell to use for execution command (default: /bin/bash -e or /bin/sh -e if bash is not available)
environment N Map Additional environmental variables, locally scoped to command
background N Boolean Whether or not this step should run in the background (default: false)
working_directory N String What directory to run this step in (default: working_directory of the job)
no_output_timeout N String Elasped time the command can run without output. The string is a decimal with unit suffix, such as “20m”, “1.25h”, “5s” (default: 10 minutes)
when N String When to run step and takes the following values: always, on_success, on_fail (default: on_success)

Each run declaration represents a new shell. It’s possible to specify a multi-line command, each line of which will be run in the same shell:

- run:
    command: |
      echo Running test
      mkdir -p /tmp/test-results
      make test
Default shell options

Our default shell has a few options enabled by default:

-e

Exit immediately if a pipeline (which may consist of a single simple command), a subshell command enclosed in parentheses, or one of the commands executed as part of a command list enclosed by braces exits with a non-zero status.

So if in the previous example mkdir failed to create a directory and returned a non-zero status, then command execution would be terminated, and the whole step would be marked as failed. If you desire the opposite behaviour, you need to add set +e in your command or override the default shell in your configuration map of run. For example:

- run:
    command: |
      echo Running test
      set +e
      mkdir -p /tmp/test-results
      make test

- run:
    shell: /bin/sh
    command: |
      echo Running test
      mkdir -p /tmp/test-results
      make test

-o

If pipefail is enabled, the pipeline’s return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully. The shell waits for all commands in the pipeline to terminate before returning a value.

For example:

- run: make test | tee test-output.log

If make test fails, the -o pipefail option will cause the whole step to fail. Without -o pipefail, the step will always run successfully because the result of the whole pipeline is determined by the last command (tee test-output.log), which will always return a zero status.

Note that even if make test fails the rest of pipeline will be executed.

If you want to avoid this behaviour, you can specify set +o pipefail in the command or override the whole shell (see example above).

In general, we recommend using the default options (-eo pipefail) because they show errors in intermediate commands and simplify debugging job failures. For convenience, the UI displays the used shell and all active options for each run step.

Background commands

The background attribute allows for executing commands in the background. In this case, job execution will immediately proceed to the next step rather than waiting for the command to return. While debugging background commands is more difficult, running commands in the background might be necessary in some cases. For instance, to run Selenium tests you may need to have X virtual framebuffer running:

- run:
    name: Running X virtual framebuffer
    command: Xvfb :99 -screen 0 1280x1024x24
    background: true

- run: make test
Shorthand syntax

run has a very convenient shorthand syntax:

- run: make test

# shorthanded command can also have multiple lines
- run: |
    mkdir -p /tmp/test-results
    make test

In this case, command and name become the string value of run, and the rest of the config map for that run have their default values.

Example
- run:
    name: Testing application
    command: make test
    shell: /bin/bash
    working_directory: ~/my-app
    no_output_timeout: 30m
    environment:
      FOO: "bar"

- run: echo 127.0.0.1 devhost | sudo tee -a /etc/hosts

- run: |
    sudo -u root createuser -h localhost --superuser ubuntu &&
    sudo createdb -h localhost test_db

checkout

Special step used to check out source code to the configured path (defaults to the working_directory).

Key Required Type Description
path N String Checkout directory (default: job’s working_directory)

If path already exists and is:

  • a git repo - step will not clone whole repo, instead will pull origin
  • NOT a git repo - step will fail.

In the case of checkout, the step type is just a string with no additional attributes:

- checkout

save_cache

Generates and stores a cache of a file or directory of files such as dependencies or source code in our object storage. Later builds can restore this cache. Learn more in the caching documentation.

Key Required Type Description
paths Y List List of directories which should be added to the cache
key Y String Unique identifier for this cache

The cache for a specific key is immutable and cannot be changed once written. NOTE: If the cache for the given key already exists it won’t be modified, and job execution will proceed to the next step.

When storing a new cache, the key value may contain special templated values for your convenience:

Template Description
{{ .Branch }} the VCS branch currently being built
{{ .BuildNum }} the CircleCI build number for this build
{{ .Revision }} the VCS revision currently being built
{{ .CheckoutKey }} the SSH key used to checkout the repo
{{ .Environment.variableName }} the environment variable variableName
{{ checksum "filename" }} a base64 encoded SHA256 hash of the given filename’s contents. This should be a file committed in your repo. Good candidates are dependency manifests, such as package.json, pom.xml or project.clj. It’s important that this file does not change between restore_cache and save_cache, otherwise the cache will be saved under a cache key different than the one used at restore_cache time.
{{ epoch }} the current time in seconds since the unix epoch.

During step execution, the templates above will be replaced by runtime values and use the resultant string as the key.

Template examples:

  • myapp-{{ checksum "package.json" }} - cache will be regenerated every time something is changed in package.json file, different branches of this project will generate the same cache key.
  • myapp-{{ .Branch }}-{{ checksum "package.json" }} - same as the previous one, but each branch will generate separate cache
  • myapp-{{ epoch }} - every build will generate separate cache

While choosing suitable templates for your cache key, keep in mind that cache saving is not a free operation, because it will take some time to upload the cache to our storage. So it make sense to have a key that generates a new cache only if something actually changed and avoid generating a new one every build.

Example
- save_cache:
    key: v1-myapp-{{ checksum "project.clj" }}
    paths:
      - /home/ubuntu/.m2

restore_cache

Restores a previously saved cache based on a key. Cache needs to have been saved first for this key using save_cache step. Learn more in the caching documentation.

Key Required Type Description
key Y (1) String Single cache key to restore
keys Y (1) List List of cache keys to lookup for a cache to restore. Only first existing key will be restored.

(1) at least one attribute has to be present. If key and keys are both given, key will be checked first, and then keys.

A key is searched against existing keys as a prefix.

NOTE: When there are multiple matches, the most recent match will be used, even if there is a more precise match.

For example:

steps:
  - save_cache:
      key: v1-myapp-cache
      paths:
        - ~/d1

  - save_cache:
      key: v1-myapp-cache-new
      paths:
        - ~/d2

  - run: rm -f ~/d1 ~/d2

  - restore_cache:
      key: v1-myapp-cache

In this case cache v1-myapp-cache-new will be restored because it’s the most recent match with v1-myapp-cache prefix even if the first key (v1-myapp-cache) has exact match.

For more information on key formatting, see the key section of save_cache step.

When CircleCI encounters a list of keys, the cache will be restored from the first one matching an existing cache. Most probably you would want to have a more specific key to be first (for example, cache for exact version of package.json file) and more generic keys after (for example, any cache for this project). If no key has a cache that exists, the step will be skipped with a warning.

A path is not required here because the cache will be restored to the location from which it was originally saved.

Example
- restore_cache:
    keys:
      - v1-myapp-{{ checksum "project.clj" }}
      # if cache for exact version of `project.clj` is not present then load any most recent one
      - v1-myapp-

# ... Steps building and testing your application ...

# cache will be saved only once for each version of `project.clj`
- save_cache:
    key: v1-myapp-{{ checksum "project.clj" }}
    paths:
      - /foo

deploy

Special step for deploying artifacts.

deploy uses the same configuration map and semantics as run step. Jobs may have more than one deploy step.

In general deploy step behaves just like run with one exception - in a build with parallelism, the deploy step will only be executed by node #0 and only if all nodes succeed. Nodes other than #0 will skip this step.

Example
- deploy:
    command: |
      if [ "${CIRCLE_BRANCH}" == "master" ]; then
        ansible-playbook site.yml
      fi

store_artifacts

Step to store artifacts (for example logs, binaries, etc) to be available in the web UI or via the API. Learn more about artifacts

Key Required Type Description
path Y String Directory in the primary container to save as build artifacts
destination N String Prefix added to the artifact paths in the artifacts API (default: the directory of the file specified in path)

There can be multiple store_artifacts steps in a job. Using a unique prefix for each step prevents them from overwriting files.

Example
- store_artifacts:
    path: /code/test-results
    destination: prefix

store_test_results

Special step used to upload test results so they can be used for timing analysis. Note At this time the results are not shown as artifacts in the web UI. To see test result as artifacts please also upload them using store_artifacts.

Key Required Type Description
path Y String Directory containing JUnit XML or Cucumber JSON test metadata files

The directory layout should match the classic CircleCI test metadata directory layout.

Example
- store_test_results:
    path: /tmp/test-results

add_ssh_keys

Special step that adds SSH keys configured in the project’s UI to the container, and configure ssh to use them.

Key Required Type Description
fingerprints N List List of fingerprints corresponding to the keys to be added (default: all keys added)
- add_ssh_keys:
    fingerprints:
      - "b7:35:a6:4e:9b:0d:6d:d4:78:1e:9a:97:2a:66:6b:be"

Note that CircleCI 2.0 builds are auto configured with ssh-agent with all keys auto-loaded, and is sufficient for most cases. add_ssh_keys may be needed to have greater control over which SSH keys to authenticate (e.g. avoid “Too many authentication failures” problem when having too many SSH keys).

Full Example

version: 2
jobs:
  build:
    docker:
      - image: ubuntu:14.04

      - image: mongo:2.6.8
        command: [mongod, --smallfiles]

      - image: postgres:9.4.1
        # some containers require setting environment variables
        environment:
          POSTGRES_USER: root

      - image: redis@sha256:54057dd7e125ca41afe526a877e8bd35ec2cdd33b9217e022ed37bdcf7d09673

      - image: rabbitmq:3.5.4

    environment:
      TEST_REPORTS: /tmp/test-reports

    working_directory: ~/my-project

    branches:
      ignore:
        - develop
        - /feature-.*/

    steps:
      - checkout

      - run: command: echo 127.0.0.1 devhost | sudo tee -a /etc/hosts

      # Create Postgres users and database
      # Note the YAML heredoc '|' for nicer formatting
      - run: |
          sudo -u root createuser -h localhost --superuser ubuntu &&
          sudo createdb -h localhost test_db

      - restore_cache:
          keys:
            - v1-my-project-{{ checksum "project.clj" }}
            - v1-my-project-

      - run:
          environment:
            SSH_TARGET: "localhost"
            TEST_ENV: "linux"
          command: |
            set -xu
            mkdir -p ${TEST_REPORTS}
            run-tests.sh
            cp out/tests/*.xml ${TEST_REPORTS}

      - run: |
          set -xu
          mkdir -p /tmp/artifacts
          create_jars.sh ${CIRCLE_BUILD_NUM}
          cp *.jar /tmp/artifacts

      - save_cache:
          key: v1-my-project-{{ checksum "project.clj" }}
          paths:
            - ~/.m2

      # Deploy staging
      - deploy:
          command: |
            if [ "${CIRCLE_BRANCH}" == "staging" ];
              then ansible-playbook site.yml -i staging;
            fi

      # Deploy production
      - deploy:
          command: |
            if [ "${CIRCLE_BRANCH}" == "master" ];
              then ansible-playbook site.yml -i production;
            fi

      # Save artifacts
      - store_artifacts:
          path: /tmp/artifacts
          destination: build

      # Upload test results
      - store_test_results:
          path: /tmp/test-reports