Pipeline Processing

Last updated
Tags Cloud Server v3.x

This document describes how your projects are processed using our pipelines engine and some of the features available to use in your pipelines. Pipelines are available on CircleCI cloud and self hosted installations of CircleCI server v3.x.

What are pipelines?

CircleCI pipelines are the highest-level unit of work, encompassing a project’s full .circleci/config.yml file. Pipelines include your workflows, which coordinate your jobs. They have a fixed, linear lifecycle, and are associated with a specific actor. Pipelines trigger when a change is pushed to a project that has a CircleCI configuration file included, and can also be scheduled, triggered manually through the CircleCI app, or using the API.

Pipelines are not available on installations of CircleCI server v2.x.

The following features are available for use in your pipelines:

  • Trigger a pipeline via the API with the trigger a pipeline endpoint.
  • Use pipeline parameters to trigger conditional workflows.
  • Use version 2.1 config, which provides access to:
    • Reusable config elements, including executors, commands and jobs.
    • Packaged reusable config, known as orbs.
    • Improved config validation error messages.
    • Option to enable auto-cancel, within Advanced Settings, to terminate workflows when new pipelines are triggered on non-default branches.

Note, it is important to carefully consider the impact of enabling the auto-cancel feature, for example, if you have configured automated deployment jobs on non-default branches.

Transitioning to pipelines

When migrating from a server v2.x to a v3.x installation you will have project configurations made before the introduction of pipelines. Pipelines are automatically enabled for server v3.x installations so all you need to do is change your project configurations (.circleci/_config.yml) to version: 2.1 to access all the features described in the section above.

Pipelines with 2.0 configuration

When using CircleCI cloud or server v3.x the CircleCI pipelines engine is automatically enabled. If, for whatever reason, you continue to use a 2.0 config, CircleCI will inject the CIRCLE_COMPARE_URL environment variable into all jobs for backwards compatibility.

This environment variable is generated in a different way compared to the version available in legacy jobs, and is not always available – it is not injected when there is no meaningful previous revision, for example, on the first push of commits to an empty repository, or when a new branch is created/pushed without any additional commits.

See also

Refer to the Skipping and Cancelling Builds document for more details.

Help make this document better

This guide, as well as the rest of our docs, are open-source and available on GitHub. We welcome your contributions.