Orchestration cookbook

Cloud

This cookbook contains "recipes" for common pipeline, workflow and job execution goals and patterns.

Trigger a pipeline when a PR is marked ready for review

Goal

You may want to run a set of high credit-consuming tests only when a pull request is ready to be taken out of "draft" — marked as ready for review. To do this, set up a trigger for your pipeline that triggers when a pull request is marked ready for review, as follows:

Steps

Set up a GitHub App pipeline you want to trigger when a PR is marked ready for review. Steps for setting up a pipeline are available in the Pipelines and triggers overview page.

Figure 1. Add pipeline options for GitHub App
Set up a trigger for your pipeline and select PR marked ready for review under the Run on menu. Steps for setting up a trigger are available in the Pipelines and triggers overview page.

Figure 2. Run on trigger options for GitHub App triggers

Notes

This option is only available for GitHub App pipelines. If you have a GitHub OAuth integration you can install the GitHub App into your GitHub org and then create GitHub App pipelines. This is a quick one-time action, as described in the GitHub integration page.

To find out which GitHub integration type you have, check your Organization slug at Organization settings Overview. OAuth authenticated orgs are structured as follows:

github/<your-org-name> or gh/<your-org-name>
bitbucket/<your-org-name> or bb/<your-org-name>

An organization slug for a GitHub App integration is in the following format:

circleci/<UID>

Trigger a pipeline when a PR is merged

Goal

You may want to trigger a specific pipeline when a pull request is merged, for example, to run clean-up/teardown scripts. To do this, set up a trigger for a "clean-up" pipeline that is triggered when a pull request is merged.

Steps

Set up a GitHub App pipeline you want to trigger when a PR is merged. Steps for setting up a pipeline are available in the Pipelines and triggers overview page.

Figure 3. Add pipeline options for GitHub App
Set up a trigger for your pipeline and select PR merged under the Run on menu. Steps for setting up a trigger are available in the Pipelines and triggers overview page.

Figure 4. Run on trigger options for GitHub App triggers

Notes

To find out which GitHub integration type you have, check your Organization slug at Organization settings Overview. OAuth authenticated orgs are structured as follows:

github/<your-org-name> or gh/<your-org-name>
bitbucket/<your-org-name> or bb/<your-org-name>

An organization slug for a GitHub App integration is in the following format:

circleci/<UID>

Run a workflow only when a pull request is opened

Goal

You may want to configure a workflow in your pipeline that will only run on a pipeline that is triggered when a pull request is opened. To do this you can use a workflow filter.

Steps

Update your CircleCI config to include a workflow that you only want to run when a PR is opened.

workflows:
  workflow-pr-open:
    # Only run this workflow when a PR is opened
    when: pipeline.event.name == "pull_request" and pipeline.event.action == "opened"
    jobs:
      - my-job-1
      - my-job-2

Notes

The pipeline.event.name and pipeline.event.action variables are only available for GitHub App pipelines. If you have a GitHub OAuth integration you can install the GitHub App into your GitHub org and then create GitHub App pipelines. This is a quick one-time action, as described in the GitHub integration page.

To find out which GitHub integration type you have, check your Organization slug at Organization settings Overview. OAuth authenticated orgs are structured as follows:

github/<your-org-name> or gh/<your-org-name>
bitbucket/<your-org-name> or bb/<your-org-name>

An organization slug for a GitHub App integration is in the following format:

circleci/<UID>

Run a workflow only for specific branches

Goal

You may want to configure a workflow that will only run when the pipeline is triggered on a specific branch.

Steps

Update your CircleCI config to include a workflow that will only run for specific branches.

workflows:
  nightly-run-workflow:
# The "nightly-run-workflow" workflow only runs if the branch name is "main" OR "staging".
    when: pipeline.git.branch == "main" or pipeline.git.branch == "staging"
    jobs:
      - build
      - deploy:
          requires:
            - build

Notes

Environment variables cannot be used in logic statements. You can only use pipeline values or pipeline parameters.

Logic statements are evaluated at compilation time whereas the actual values of environment variables only become available at run time.

See the "Can I use environment variables in conditional statements?" article for more information.

Alternatively, you can use the classic/legacy syntax to achieve the same result:

workflows:
  my-workflow:
# The "my-workflow" workflow only runs if the branch name is "main" OR "staging".
    when:
      or:
        - equal: [ main, << pipeline.git.branch >> ]
        - equal: [ staging, << pipeline.git.branch >> ]

You can also choose to prevent the workflow from running for specific branches:

workflows:
  my-workflow:
# The "my-workflow" workflow only runs if the branch name is not "feature".
    when:
    not:
      matches:
        pattern: "^feature$"
        value: << pipeline.git.branch >>

To trigger a pipeline and pass custom values for your pipeline parameters, you can either:

Run a workflow depending on the value of a boolean parameter

Goal

You may want to configure a workflow that will only run when a boolean pipeline parameter is true.

Steps

Update config to declare a pipeline parameter with a default value of false

Update you workflow config to use a when statement so the workflow will only run when the pipeline parameter is true

workflows:
  integration_tests:
# The "integration_tests" workflow only runs if the pipeline parameter "run_integration_tests" is true.
    when: pipeline.parameters.run_integration_tests == true
    jobs:
      - mytestjob

Notes

Environment variables cannot be used in logic statements. You can only use pipeline values or pipeline parameters.

Logic statements are evaluated at compilation time whereas the actual values of environment variables only become available at run time.

See the "Can I use environment variables in conditional statements?" article for more information.

You can also use the shorthand syntax to achieve the same result:

version: 2.1

parameters:
  run_integration_tests:
    type: boolean
    default: false

workflows:
  integration_tests:
# The "integration_tests" workflow only runs if the pipeline parameter "run_integration_tests" is true.
    when: << pipeline.parameters.run_integration_tests >>
    jobs:
      - mytestjob

You can create complex logic statements that involve multiple parameters:

workflows:
  conditional-workflow:
# The "conditional-workflow" workflow only runs if the branch is "main" AND the pipeline parameter "param1" is false AND either the pipeline parameter "param1" is true OR the pipeline parameter "param2" is true.
    when:
      and:
        - equal: [ main, << pipeline.git.branch >> ]
        - not: << pipeline.parameters.param1 >>
        - or: [ << pipeline.parameters.param1 >>, << pipeline.parameters.param2 >> ]

To trigger a pipeline and pass custom values for your pipeline parameters, you can either:

Run a step depending on the output of a nested logic statement

Goal

You may want to configure a step that will run depending on the output of a nested logic statement.

Steps

Update your CircleCI config to use the when step.

version: 2.1

parameters:
  param1:
    type: boolean
    default: false
  param2:
    type: boolean
    default: false
  param3:
    type: boolean
    default: false
  param4:
    type: boolean
    default: true

jobs:
  test:
    docker:
      - image: cimg/node:19.0.1
    steps:
      - checkout
      - when:
          condition:
            and:
              - or:
                - and:
                  - equal: [ main, << pipeline.git.branch >> ]
                  - equal: [ false, << pipeline.parameters.param1 >> ]
                - not: << pipeline.parameters.param3 >>
              - or:
                - equal: [ false, << pipeline.parameters.param3 >> ]
                - or: [ << pipeline.parameters.param1 >>, << pipeline.parameters.param2 >> ]  
# The output of the nested logic statement will depend on the values of the pipeline parameters. This step will only run if the condition evaluates as true.
          steps:
            - run: echo "The condition evaluated as true"

Notes

The logic statement can be nested to a maximum depth of 100 levels.
To trigger a pipeline and pass custom values for your pipeline parameters, you can either:

Run a job only on specific branches or when the value of a pipeline parameter is true

Goal

You may want to configure a job that only runs on specific branches or when the value of a pipeline parameter is true when triggered via the API.

Steps

Update your CircleCI config to include an expression-based job filter.

version: 2.1

parameters:
  run-storybook-tests:
    type: boolean
    default: false

...
# jobs configuration ommitted for brevity

workflows:
  build:
    jobs:
      - setup
      - storybook-tests:
# The "storybook-tests" job only runs if the "setup" job completes successfully AND if either the "run-storybook-test"s pipeline parameter is true OR the branch name is "dry-run-deploy" OR the branch name starts with "deploy".
          requires:
            - setup
          filters: |
            pipeline.parameters.run-storybook-tests
            or pipeline.git.branch == "dry-run-deploy"
            or pipeline.git.branch starts-with "deploy"

Notes

Environment variables cannot be used in logic statements. You can only use pipeline values or pipeline parameters.

Logic statements are evaluated at compilation time whereas the actual values of environment variables only become available at run time.

See the "Can I use environment variables in conditional statements?" article for more information.
To trigger a pipeline and pass custom values for your pipeline parameters, you can either:

Run a workflow only for scheduled pipelines with a specific name

Goal

In some cases, you may want to configure a workflow that will only run when a scheduled pipeline has a specific name.

Steps

Update your CircleCI config to use the pipeline.scheduled_source and pipeline.schedule.name pipeline values.

workflows:
  nightly-run-workflow:
# The "nightly-run-workflow" workflow only runs if the branch name is "main" OR "staging".
    when: pipeline.git.branch == "main" or pipeline.git.branch == "staging"
    jobs:
      - build
      - deploy:
          requires:
            - build

Notes

Environment variables cannot be used in logic statements. You can only use pipeline values or pipeline parameters.

Logic statements are evaluated at compilation time whereas the actual values of environment variables only become available at run time.

See the "Can I use environment variables in conditional statements?" article for more information.

Run a workflow when open pull requests are modified

Goal

You may want to have a workflow run when a open pull requests are updated with new commits (synchronize) or labeled.

Steps

Update your CircleCI config to use the pipeline.event.name and pipeline.event.action pipeline values.

workflows:
# The "run_on_pr" workflow only runs when open pull requests are labeled or updated with new commits.
  run_on_pr:
    when: (pipeline.event.name == "pull_request" and pipeline.event.action == "labeled") or (pipeline.event.name == "pull_request" and pipeline.event.action == "synchronize")

Notes

The pipeline.event.name and pipeline.event.action variables are only available for GitHub App pipelines. If you have a GitHub OAuth integration you can install the GitHub App into your GitHub org and then create GitHub App pipelines. This is a quick one-time action, as described in the GitHub integration page.
Environment variables cannot be used in logic statements. You can only use pipeline values or pipeline parameters.

Logic statements are evaluated at compilation time whereas the actual values of environment variables only become available at run time.

See the "Can I use environment variables in conditional statements?" article for more information.

Use no-op jobs to create a cleaner workflow graph

Goal

You may want to organize your workflow graph to make it more readable without adding additional build time or consuming credits. Using no-op jobs can help create a cleaner structure for complex workflows with many dependencies.

The following snippet shows how your workflow configuration might look without using a no-op job:

version: 2.1

jobs:
  build:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm ci
      - run: npm run build

  test-unit:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run test:unit

  test-integration:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run test:integration

  test-e2e:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run test:e2e

  deploy:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run deploy

workflows:
  build-test-deploy:
    jobs:
      - build
      - test-unit:
          requires:
            - build
      - test-integration:
          requires:
            - build
      - test-e2e:
          requires:
            - build
      - deploy:
          requires:
            - test-unit
            - test-integration
            - test-e2e

Notice how the deploy job needs to explicitly list all three test jobs in its requires section. As your workflow grows, this list could become longer and more difficult to maintain.

Here is the workflow graph for this config:

Workflow graph showing snippet without `no-op` job

Steps

Add a no-op job to your CircleCI configuration file.

jobs:
  test-coordinator:
    type: no-op

  test-unit:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run test:unit

  test-integration:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run test:integration

  test-e2e:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run test:e2e

Configure your workflow to use the no-op job as a coordinator for downstream jobs.

workflows:
  build-test-deploy:
    jobs:
      - build
      - test-unit:
          requires:
            - build
      - test-integration:
          requires:
            - build
      - test-e2e:
          requires:
            - build
      - test-coordinator:
          requires:
            - test-unit
            - test-integration
            - test-e2e
      - deploy:
          requires:
            - test-coordinator

The workflow graph will now look like this:

Workflow graph showing snippet with `no-op` job

Notes

The no-op job performs no actions and consumes no credits. This makes it an ideal structural element in your workflow.
This pattern is useful when you want to visually group related jobs in the CircleCI UI.
Using no-op jobs can simplify workflow management when dealing with complex dependency chains.
Without the test-coordinator job, the deploy job would need to list all test jobs in its requires section.

Simplify GitHub required status checks with no-op jobs

Goal

You may want to simplify the management of GitHub required status checks for your branch protection rules. Use a single no-op job as a gatekeeper that depends on all your critical checks. This approach is means you only need to manage a single required status check rather than multiple status checks in GitHub.

The following snippet shows how your workflow configuration might look without using a no-op job:

version: 2.1

jobs:
  lint:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run lint

  test:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm test

  security-scan:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run security-check

workflows:
  main:
    jobs:
      - lint
      - test
      - security-scan

With this configuration, in GitHub you would need to set up branch protection rules for each job (lint, test, security-scan). You would need to update both CircleCI config and GitHub settings when adding more jobs.

Steps

Add a no-op job to your CircleCI configuration file that will serve as your single required status check. In this example the no-op job is all-tests-passed:

jobs:
  all-tests-passed:
    type: no-op

  lint:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run lint

  test:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm test

  security-scan:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run security-check

Configure your workflow to make all critical jobs prerequisites for the no-op job. The all-tests-passed will only succeed if all its required jobs succeed.

workflows:
  main:
    jobs:
      - lint
      - test
      - security-scan
      - all-tests-passed:
          requires:
            - lint
            - test
            - security-scan

In GitHub, set up a branch protection rule to require only the all-tests-passed job as a required status check. See the GitHub docs for more information on branch protection rules.

Notes

This approach simplifies branch protection management in GitHub by requiring only one status check.
When you add new checks, you only need to update the requires list for your no-op job.
The no-op job will succeed only if all its required jobs succeed.
This pattern helps keep GitHub and CircleCI configurations in sync without manual updates.

Suggest an edit to this page

Make a contribution

Learn how to contribute

Still need help?

Ask the CircleCI community

Join the research community

Visit our Support site

Orchestration cookbook

On This Page

Trigger a pipeline when a PR is marked ready for review

Goal

Steps

Notes

Trigger a pipeline when a PR is merged

Goal

Steps

Notes

Run a workflow only when a pull request is opened

Goal

Steps

Notes

Run a workflow only for specific branches

Goal

Steps

Notes

Run a workflow depending on the value of a boolean parameter

Goal

Steps

Notes

Run a step depending on the output of a nested logic statement

Goal

Steps

Notes

Run a job only on specific branches or when the value of a pipeline parameter is true

Goal

Steps

Notes

Run a workflow only for scheduled pipelines with a specific name

Goal

Steps

Notes

Run a workflow when open pull requests are modified

Goal

Steps

Notes

Use no-op jobs to create a cleaner workflow graph

Goal

Steps

Notes

Simplify GitHub required status checks with no-op jobs

Goal

Steps

Notes

Suggest an edit to this page

Still need help?