Pipelines overview

Cloud

Server v4.x

Server v3.x

CircleCI pipelines are the highest-level unit of work, and they are described by a configuration file, for example, for example, .circleci/config.yml. Triggers can be set up to automate when a pipeline will be triggered. Pipelines coordinate the execution of workflows and jobs to build, test, and deploy your code. Workflows can be conditional, based on filters, job requirements, and pipeline parameters.

Pipelines include your workflows, which coordinate your jobs. Pipelines have a fixed, linear lifecycle, and are associated with a specific actor. Pipelines trigger when a change is pushed to a project that has a CircleCI configuration file included, and can also be scheduled, triggered manually through the CircleCI app, or using the API.

Dashboard

When visiting your CircleCI dashboard, you are shown a list of recently-triggered pipelines for your organization.

Pipeline architecture

A pipeline is composed of workflows, which are composed of jobs. By navigating from a pipeline to a specific job, you can access your job output, test results, and artifacts through several tabs.

The output of each job can be opened in a new tab (in either raw or formatted styling) with a unique link, making it shareable between team members.

Configuration sources and project triggers

The information in this section on configuration sources and triggers applies to accounts integrated with the CircleCI GitHub App or GitLab. There are two GitHub integration types, either OAuth or through the GitHub App. To find out which GitHub integration type you have refer to the GitHub App integration page.

Set up configuration sources and project triggers to define when and how pipelines are automatically triggered for a project. These triggers may be from external services (custom webhooks), or from commits or tags on a version control system.

Configuration source

Configuration sources define the units of work that can be run within your projects. Each configuration source defines the location of a pipeline configuration file, and can be set up to be used when triggering a pipeline. Configuration sources can be associated with one or more triggers.

A project can have one or more configuration sources. The default location for a CircleCI configuration file is .circleci/config.yml. If you authenticated through the GitHub App, you can set the location to any path to a *.yml file.

When you create a project in CircleCI and connect it to a repository, a configuration source is set up for you automatically, and connected to a configuration file. The connected configuration file will be either a file you commit yourself or one generated by CircleCI based on the code in your project.

Add a configuration source

If you created a project in the CircleCI web app you will find a configuration source is already set up.

To add a configuration source, follow these steps:

In the CircleCI web app, select Projects in the sidebar.
Select the ellipsis next to your project () and choose Project Settings.
Select Configuration in the sidebar.
Select Add Configuration Source.

Complete the form fields and options:

Give your configuration source a descriptive name, for example Run tests and deploy.

Under Configuration Source select the platform that matches where your project is set up (for example, GitHub App).

For GitLab Self-managed, you can input any instance that you have previously set up in CircleCI. If you wish to set a different feature branch or repository from a self-managed instance as a new configuration source, you will first need to add a new connection via your Integrations. In either case, you will also need to enter your personal access token again to authorize this connection.

Authorize your connection if this is not already showing with a

Select your repository

Select the same repository for your configuration source and trigger (described in the following section). There is currently a bug that means that CircleCI will always look for the configuration source in the same repository from which the triggering event was emitted.

Select a fallback branch to use in case the triggering event does not have a triggering branch. This branch will be used if you trigger this pipeline via a custom webhook.
Enter the path to your configuration file relative to the root of your repository. For example, .circleci/config.yml. If you are integrated through the GitHub App, you can enter any path to a .yml file, and the file can have any name you choose.

Select Save.

Once you have set up a configuration source you need to set up a project trigger to connect it to. This is described in the following section.

Project triggers

If you created a project in the CircleCI web app you will find a trigger is already set up.

Triggers are rules to define that a configuration source should be run on a given event. A project can have one or more triggers, which are associated with events from a source of change. The most common type of trigger is a VCS event, for example a push to a branch on a GitHub repository, but any system that can emit outbound webhooks can also be set up as a pipeline trigger. A trigger can have only one configuration source but a configuration source can have multiple triggers.

A trigger kicks off a pipeline to run the workflows defined by your connected configuration source.

Add a project trigger

To add a project trigger for a configuration source, follow these steps:

When a trigger is created, CircleCI registers a webhook with GitLab. Push events from GitLab are sent to CircleCI. CircleCI then uses the event data to determine if a pipeline should run, and if so, which pipeline should be run.

In addition to a configuration source, each trigger includes the webhook URL, and in this scenario, a CircleCI-created GitLab token. The webhook URL and GitLab token are used to securely register the webhook within GitLab in order to receive push events from your GitLab repository. You can view webhooks for a project in GitLab at Settings Webhooks.

To add a project trigger for a configuration source, follow these steps:

In the CircleCI web app select Projects in the sidebar
Find your project in the list, select the ellipsis () next to it and choose Project Settings.
Select Triggers in the sidebar.
Select Add Trigger.
Select the same location in the "Connect to" dropdown menu that you selected for your configuration source (for example, GitLab).
Select Next.

Complete the form fields and options:

Give your trigger a descriptive name.

Authorize your connection if this is not already showing with a (Not required for custom webhooks).

For GitLab self-managed you can enter the URL for an instance you have previously set up with CircleCI. You will need to enter the relevant personal access token again here to authorize the connection.

Select your repository from the dropdown menu. This should match the repository your configuration source is connected to (Not required for custom webhooks).
Choose your configuration source from the "Choose config to run" menu.
(Optional) You can configure trigger filters.

Select Save

When setting up a trigger you will see a modal titled Webhook URL requesting that you set up a webhook in GitLab. You do not need to take action. The webhook is set up automatically by CircleCI. This is a known issue and will be fixed.

To verify your trigger is set up correctly, trigger an event from your repository.

Trigger filters allow you to determine when a trigger should initiate a build based on the parameters provided by Gitlab’s webhook. CircleCI provides some common options, for example, only build on merge requests, but you can also build your own rules using the custom filter option. For example, a custom filter would allow you to only build on a specific branch or user.

VS Code extension

If you use Visual Studio Code, you can also monitor and interact with your pipelines directly from VS Code with the official CircleCI extension. The extension allows you to customize which projects and pipelines you want to follow, as well as view job logs and test results, download artifacts, approve, re-run, and debug jobs with SSH, and get notified when your workflows fail or need approval.