How to use the CircleCI local CLI

1 month ago5 min read

Cloud

Server v4.x

Server v3.x

This page describes how to use some features of the CircleCI CLI.

Prerequisites

To follow the guides on this page you will need the following:

A CircleCI account (you can sign up for free)
Local installation of the CircleCI CLI. Steps and options are available on the Installing the local CLI page.
Set up and configure the CircleCI CLI.

Validate a CircleCI config

Using VS Code? You can validate you CircleCI configuration using the VS Code extension. For full information about the features provided in the VS Code extension, see the VS Code extension overview page.

You can avoid pushing additional commits to test your .circleci/config.yml by using the CLI to validate your configuration locally. Follow these steps:

Navigate to your project directory. You can a directory with a .circleci/config.yml file
Run the following, providing the path to your .circleci/config.yml:
```
circleci config validate <path-to-config.yml>
```

If your configuration is valid, you should see the following:

Config file at .circleci/config.yml is valid

Otherwise you will see errors, for example:

Error: config compilation contains errors: config compilation contains errors:
	- Unable to parse YAML
	- mapping values are not allowed here
	-  in 'string', line 20, column 22:
	-               environment:
	-                          ^

Orb development kit

The orb development kit refers to a suite of tools that work together to simplify the orb development process, with automatic testing and deployment on CircleCI. There are two commands in the CLI that are a part of the orb development kit:

The following command initializes a new orb project:
```
circleci orb init
```
The following command packs an orb with local scripts:
```
circleci orb pack
```

For more information on orb packing, see the Orbs Concepts page.

Validate an orb in your configuration file

You can validate your orb with the following:

circleci orb validate <path-to-your-orb.yml>

Packing a config

circleci config pack

This CLI pack command (separate to circleci orb pack described above) allows you to create a single YAML file from several separate files (based on directory structure and file contents). The pack command implements FYAML, a scheme for breaking YAML documents across files in a directory tree. This is useful for breaking up source code for large orbs and allows custom organization of your orbs' YAML configuration.

How you name and organize your files when using the pack command will determine the final orb.yml output. Consider the following folder structure example:

When packing a config using circleci config pack, the component names are not included in the configuration files, they are taken from the file names.

$ tree
.
└── your-orb-source
    ├── @orb.yml
    ├── commands
    │   └── foo.yml
    └── jobs
        └── bar.yml

3 directories, 3 files

The UNIX tree command is great for printing out folder structures. In the example tree structure above, the pack command will map the folder names and file names to YAML keys, and map the file contents as the values to those keys.

The following command will pack up the example folder from above:

circleci config pack your-orb-source

And the output will be in your .yml file:

# Contents of @orb.yml appear here
commands:
  foo:
    # contents of foo.yml appear here
jobs:
  bar:
    # contents of bar.yml appear here

Other configuration packing capabilities

A file beginning with @ will have its contents merged into its parent folder level. This can be useful at the top level of an orb, when one might want generic orb.yml to contain metadata, but not to map into an orb key-value pair.

For example, the following structure:

cat foo/bar/@baz.yml
\{baz: qux}

Maps to:

bar:
  baz: qux

Processing a config

Running the following command validates your config, but will also display expanded source configuration alongside your original configuration. This is particularly useful if you are using orbs:

circleci config process <path-to-config.yml>

Consider the following example configuration that uses the node orb:

version: 2.1

orbs:
  node: circleci/node@4.7.0

workflows:
  example-workflow:
      jobs:
        - node/test

Processing this config file will output a YAML file like the example below. This is the expanded source configuration using version: 2 syntax. All version: 2.1 elements are processed:

# Orb 'circleci/node@4.7.0' resolved to 'circleci/node@4.7.0'
version: 2
jobs:
  node/test:
    docker:
    - image: cimg/node:13.11.0
    steps:
    - checkout
    - run:
        command: |
          if [ ! -f "package.json" ]; then
            echo
            echo "---"
            echo "Unable to find your package.json file. Did you forget to set the app-dir parameter?"
            echo "---"
            echo
            echo "Current directory: $(pwd)"
            echo
            echo
            echo "List directory: "
            echo
            ls
            exit 1
          fi
        name: Checking for package.json
        working_directory: ~/project
    - run:
        command: |
          if [ -f "package-lock.json" ]; then
            echo "Found package-lock.json file, assuming lockfile"
            ln package-lock.json /tmp/node-project-lockfile
          elif [ -f "npm-shrinkwrap.json" ]; then
            echo "Found npm-shrinkwrap.json file, assuming lockfile"
            ln npm-shrinkwrap.json /tmp/node-project-lockfile
          elif [ -f "yarn.lock" ]; then
            echo "Found yarn.lock file, assuming lockfile"
            ln yarn.lock /tmp/node-project-lockfile
          fi
          ln package.json /tmp/node-project-package.json
        name: Determine lockfile
        working_directory: ~/project
    - restore_cache:
        keys:
        - node-deps-{{ arch }}-v1-{{ .Branch }}-{{ checksum "/tmp/node-project-package.json" }}-{{ checksum "/tmp/node-project-lockfile" }}
        - node-deps-{{ arch }}-v1-{{ .Branch }}-{{ checksum "/tmp/node-project-package.json" }}-
        - node-deps-{{ arch }}-v1-{{ .Branch }}-
    - run:
        command: "if [[ ! -z \"\" ]]; then\n  echo \"Running override package installation command:\"\n  \nelse\n  npm ci\nfi\n"
        name: Installing NPM packages
        working_directory: ~/project
    - save_cache:
        key: node-deps-{{ arch }}-v1-{{ .Branch }}-{{ checksum "/tmp/node-project-package.json" }}-{{ checksum "/tmp/node-project-lockfile" }}
        paths:
        - ~/.npm
    - run:
        command: npm run test
        name: Run NPM Tests
        working_directory: ~/project
workflows:
  version: 2
  example-workflow:
    jobs:
    - node/test

Run a job in a container on your machine

The CircleCI CLI enables you to run a job from your configuration locally with Docker. This can be useful for the following:

To run tests before pushing configuration changes
Debugging your build process without impacting your build queue

Only single jobs can be run locally, not workflows.

Prerequisites

You will need to have Docker installed on your system, as well as the most recent version of the CLI. You will also need to have a project that includes a valid .circleci/config.yml file.

Run a job

Navigate to the root of your project containing the .circleci/config.yml file.
Run the following command specifying the job you would like to run. If your CircleCI configuration is set to version 2.1, you must first export your configuration to process.yml, and specify it when executing with the following commands:
```
circleci config process .circleci/config.yml > process.yml
circleci local execute -c process.yml <job-name>
```
If you are using version: 2 configuration, you can simply run:
```
circleci local execute <job-name>
```

The commands above will run the job you specify by name. The CLI uses Docker to pull down the requirements for the build and then execute your CI steps locally.

Limitations of running jobs locally

Although running jobs locally with circleci is helpful, there are some limitations.

Executors

The CLI does not support running jobs that use a machine (machine) or macOS (macos) executor locally. This is because these executors require running an additional virtual machine. Only jobs that use a Docker (docker) executor can be run locally.

Add SSH keys

It is currently not possible to add SSH keys using the add_ssh_keys CLI command.

Workflows

The CLI tool does not provide support for running workflows. By nature, workflows leverage running jobs concurrently on multiple machines allowing you to achieve faster, more complex builds. Because the CLI is only running on your machine, it can only run single jobs (which make up parts of a workflow).

Caching and online-only commands

Caching is not currently supported in local jobs. When you have either a save_cache or restore_cache step in your config, circleci will skip them and display a warning.

Further, not all commands may work on your local machine as they do online. For example, the Golang build reference above runs a store_artifacts step, however, local builds will not upload artifacts. If a step is not available on a local build you will see an error in the console.

Environment variables

For security reasons, encrypted environment variables configured in the web application will not be imported into local builds. As an alternative, you can specify environment variables to the CLI with the -e flag. See the output of the following command for more information.

circleci help build

If you have multiple environment variables, you must use the flag for each variable, for example:

circleci build -e VAR1=FOO -e VAR2=BAR

Test splitting

The CircleCI CLI is used for some advanced features during job runs, for example test splitting for build time optimization.

Context management

Contexts provide a mechanism for securing and sharing environment variables across projects. While contexts have been traditionally managed on the CircleCI web application, the CircleCI CLI provides an alternative method for managing the usage of contexts in your projects. With the CLI, you can execute several context-oriented commands:

create - Create a new context
delete - Delete the named context
list - List all contexts
remove-secret - Remove an environment variable from the named context
show - Show a context
store-secret - Store a new environment variable in the named context

The above list are "sub-commands" in the CLI, which would be executed like so:

circleci context create --org-id <org-id> <context-name> [flags]

Refer to the CLI docs for full details for each command. Many commands require that you include additional information as indicated by parameters delimited by < >. For example, when running circleci context create, you will need to provide a name for the context and your org ID.

To find your organization/user ID, select Organization Settings in the CircleCI web app side bar.

screenshot showing where to find your organization ID

Config policy management

The CircleCI CLI can be used to manage config policies for your projects. Config policies allow you to enforce best practices and standards across your organization.

For full details on creating, testing, and managing config policies for your organization, including how-to guides, see the Manage config policies section.