Documentation structure for LLMs (llms.txt)

Orchestration cookbook

Cloud

This cookbook contains "recipes" for common pipeline, workflow and job execution goals and patterns.

Trigger a pipeline when a PR is marked ready for review

Goal

You may want to run a set of high credit-consuming tests only when a pull request is ready to be taken out of "draft" — marked as ready for review. To do this, set up a trigger for your pipeline that triggers when a pull request is marked ready for review, as follows:

Steps

  1. Set up a GitHub App pipeline you want to trigger when a PR is marked ready for review. Steps for setting up a pipeline are available in the Pipelines Overview page.

  2. Set up a trigger for your pipeline and select PR marked ready for review under the Run on menu. Steps for setting up a trigger are available in the Set up Triggers page.

    Run on trigger options for GitHub App triggers
    Figure 1. Run on trigger options for GitHub App triggers

Notes

  • This option is only available for GitHub App pipelines. If you have a GitHub OAuth integration you can install the GitHub App into your GitHub org and then create GitHub App pipelines. This is a quick one-time action, as described in the Using the CircleCI GitHub App in an OAuth Organization page.

    To find out which GitHub integration type you have, check your Organization slug at Organization settings  Overview. OAuth authenticated orgs are structured as follows:

    • github/<your-org-name> or gh/<your-org-name>

    • bitbucket/<your-org-name> or bb/<your-org-name>

    An organization slug for a GitHub App integration is in the following format:

    • circleci/<UID>

Trigger a pipeline when a PR is merged

Goal

You may want to trigger a specific pipeline when a pull request is merged, for example, to run clean-up/teardown scripts. To do this, set up a trigger for a "clean-up" pipeline that is triggered when a pull request is merged.

Steps

  1. Set up a GitHub App pipeline you want to trigger when a PR is merged. Steps for setting up a pipeline are available in the Pipelines and Triggers Overview page.

  2. Set up a trigger for your pipeline and select PR merged under the Run on menu. Steps for setting up a trigger are available in the Set up Triggers page.

    Run on trigger options for GitHub App triggers
    Figure 2. Run on trigger options for GitHub App triggers

Notes

  • This option is only available for GitHub App pipelines. If you have a GitHub OAuth integration you can install the GitHub App into your GitHub org and then create GitHub App pipelines. This is a quick one-time action, as described in the Using the CircleCI GitHub App in an OAuth Organization page.

    To find out which GitHub integration type you have, check your Organization slug at Organization settings  Overview. OAuth authenticated orgs are structured as follows:

    • github/<your-org-name> or gh/<your-org-name>

    • bitbucket/<your-org-name> or bb/<your-org-name>

    An organization slug for a GitHub App integration is in the following format:

    • circleci/<UID>

Run a workflow only when a pull request is opened

Goal

You may want to configure a workflow in your pipeline that will only run on a pipeline that is triggered when a pull request is opened. To do this you can use a workflow filter.

Steps

  • Update your CircleCI config to include a workflow that you only want to run when a PR is opened.

    workflows:
      workflow-pr-open:
        # Only run this workflow when a PR is opened
        when: pipeline.event.name == "pull_request" and pipeline.event.action == "opened"
        jobs:
          - my-job-1
          - my-job-2

Notes

  • The pipeline.event.name and pipeline.event.action variables are only available for GitHub App pipelines. If you have a GitHub OAuth integration you can install the GitHub App into your GitHub org and then create GitHub App pipelines. This is a quick one-time action, as described in the Using the CircleCI GitHub App in an OAuth Organization page.

    To find out which GitHub integration type you have, check your Organization slug at Organization settings  Overview. OAuth authenticated orgs are structured as follows:

    • github/<your-org-name> or gh/<your-org-name>

    • bitbucket/<your-org-name> or bb/<your-org-name>

    An organization slug for a GitHub App integration is in the following format:

    • circleci/<UID>

Run a workflow only for specific branches

Goal

You may want to configure a workflow that will only run when the pipeline is triggered on a specific branch.

Steps

  • Update your CircleCI config to include a workflow that will only run for specific branches. You can configure the workflow using an expression.

    The nightly-run-workflow only runs on the main and staging branches
    workflows:
      nightly-run-workflow:
    # The "nightly-run-workflow" workflow only runs if the branch name is "main" OR "staging".
        when: pipeline.git.branch == "main" or pipeline.git.branch == "staging"
        jobs:
          - build
          - deploy:
              requires:
                - build

Notes

  • Environment variables cannot be used in expressions and logic statements. You can only use pipeline values or pipeline parameters.

    Expressions and logic statements are evaluated at compilation time whereas the actual values of environment variables only become available at run time.

    See the "Can I use environment variables in conditional statements?" article for more information.

  • You can also choose to prevent the workflow from running for specific branches using the not operator with matches for pattern matching:

    workflows:
      my-workflow:
    # The "my-workflow" workflow will NOT run when the branch name matches the pattern "feature-.*"
        when: not (pipeline.git.branch matches /^feature-.*$/)
        jobs:
          - build
  • To trigger a pipeline and pass custom values for your pipeline parameters, you can either:

Run a workflow depending on the value of a boolean parameter

Goal

You may want to configure a workflow that will only run when a boolean pipeline parameter is true.

Steps

  1. Update config to declare a pipeline parameter with a default value of false.

  2. Update your workflow config to use a when statement so the workflow will only run when the pipeline parameter is true.

    version: 2.1
    
    parameters:
      run_integration_tests:
        type: boolean
        default: false
    
    workflows:
      integration_tests:
    # The "integration_tests" workflow only runs if the pipeline parameter "run_integration_tests" is true.
        when: pipeline.parameters.run_integration_tests
        jobs:
          - mytestjob

Notes

  • Environment variables cannot be used in expressions and logic statements. You can only use pipeline values or pipeline parameters.

    Expressions and logic statements are evaluated at compilation time whereas the actual values of environment variables only become available at run time.

    See the "Can I use environment variables in conditional statements?" article for more information.

  • You can also use the shorthand syntax to achieve the same result:

    version: 2.1
    
    parameters:
      run_integration_tests:
        type: boolean
        default: false
    
    workflows:
      integration_tests:
    # The "integration_tests" workflow only runs if the pipeline parameter "run_integration_tests" is true.
        when: << pipeline.parameters.run_integration_tests >>
        jobs:
          - mytestjob
  • You can create complex expressions that involve multiple parameters using and, or, and parentheses for grouping:

    workflows:
      my-workflow:
    # This workflow runs when: branch is "main" AND param1 is false AND (param2 OR param3 is true)
        when: pipeline.git.branch == "main" and not pipeline.parameters.param1 and (pipeline.parameters.param2 or pipeline.parameters.param3)
        jobs:
          - build
  • To trigger a pipeline and pass custom values for your pipeline parameters, you can either:

Combine pipeline parameters with event filters in workflow conditions

Goal

You may want to configure a workflow that runs when a specific event occurs (such as a push or pull request) or when a pipeline parameter is set to true. To combine pipeline parameters with event filters in a workflow when clause, write the condition as a direct expression without << >> delimiters.

The when clause accepts an expression as its argument. The expression is evaluated to a boolean value at configuration compilation time, before the workflow runs, so template replacement is not required. For more on using expressions in your configuration, see Using Expressions in Your Configuration.

Steps

  1. Define your pipeline parameters in the parameters section of your config file. In this example, deploy is used as the parameter name, but you can use any parameter name you define:

    version: 2.1
    
    parameters:
      deploy:
        type: boolean
        default: false
  2. Write your workflow when clause as a direct expression. Reference pipeline parameters using pipeline.parameters.<name> directly (without << >> delimiters):

    The my-workflow workflow runs when: the event is a push or a pull request that is not a draft or the deploy parameter is true.
    workflows:
      my-workflow:
        when: pipeline.event.name == "push" or (pipeline.event.name == "pull_request" and pipeline.event.github.pull_request.draft == false) or pipeline.parameters.deploy
        jobs:
          - build

Notes

  • You must define your pipeline parameters in the parameters section before using them in workflow conditions. The parameter name deploy in this example is just an example. You can use any parameter name you define, for example, run_tests, skip_validation, or enable_feature.

  • The workflow when clause expects a direct expression. Use pipeline parameter names directly (for example, pipeline.parameters.deploy).

  • Do not mix << >> delimiters with direct expressions. The << >> syntax is used to mark expressions inside string values. If the when clause contains << >> delimiters, the entire value is treated as a string with template replacement rather than as a direct expression. This causes the workflow to always be created regardless of the actual condition.

    # Incorrect: mixing direct expression with << >> delimiters
    workflows:
      my-workflow:
        when: pipeline.event.name == "push" or << pipeline.parameters.deploy >>
        jobs:
          - build
  • For simple parameter checks, you can use << >> syntax on its own, since the entire value is then a single expression. Remember to replace deploy with your own parameter name:

    workflows:
      my-workflow:
        when: << pipeline.parameters.deploy >>
        jobs:
          - build
  • For more details, see the Using When in Workflows and Using Expressions in Your Configuration sections in the configuration reference.

  • To trigger a pipeline and pass custom values for your pipeline parameters, you can either:

Run a step based on a nested condition

Goal

You may want to configure a step that will run depending on the output of a nested condition using logic statement syntax.

Expressions are not supported for step-level when conditions. Use logic statement syntax for the condition field of The When Step. For workflow-level conditions, you can use Expression Syntax instead.

Steps

  • Update your CircleCI config to use The When Step with a nested logic statement.

    version: 2.1
    
    parameters:
      param1:
        type: boolean
        default: false
      param2:
        type: boolean
        default: false
      param3:
        type: boolean
        default: false
      param4:
        type: boolean
        default: true
    
    jobs:
      test:
        docker:
          - image: cimg/node:19.0.1
        steps:
          - checkout
          - when:
              condition:
                and:
                  - or:
                    - and:
                      - equal: [ main, << pipeline.git.branch >> ]
                      - equal: [ false, << pipeline.parameters.param1 >> ]
                    - not: << pipeline.parameters.param3 >>
                  - or:
                    - equal: [ false, << pipeline.parameters.param3 >> ]
                    - or: [ << pipeline.parameters.param1 >>, << pipeline.parameters.param2 >> ]  
    # The output of the nested logic statement will depend on the values of the pipeline parameters. This step will only run if the condition evaluates as true.
              steps:
                - run: echo "The condition evaluated as true"

Notes

Run a job only on specific branches or when the value of a pipeline parameter is true

Goal

You may want to configure a job that only runs on specific branches or when the value of a pipeline parameter is true.

Steps

  • Update your CircleCI config to include an Expression-Based Job Filter.

    version: 2.1
    
    parameters:
      run-storybook-tests:
        type: boolean
        default: false
    
    ...
    # jobs configuration omitted for brevity
    
    workflows:
      build:
        jobs:
          - setup
          - storybook-tests:
    # The "storybook-tests" job only runs if the "setup" job completes successfully AND if either the "run-storybook-test"s pipeline parameter is true OR the branch name is "dry-run-deploy" OR the branch name starts with "deploy".
              requires:
                - setup
              filters: |
                pipeline.parameters.run-storybook-tests
                or pipeline.git.branch == "dry-run-deploy"
                or pipeline.git.branch starts-with "deploy"

Notes

Run a workflow only on schedule triggers that have a specific name

Goal

In some cases, you may want to configure a workflow that will only run when a scheduled trigger has a specific name.

Steps

  • Update your CircleCI config to use the pipeline.trigger.type and pipeline.schedule.name Pipeline Values.

    The nightly-run-workflow only runs when the trigger is a schedule trigger AND the name of that trigger is nightly_build
    workflows:
      nightly-run-workflow:
    # The "nightly-run-workflow" workflow only runs when the trigger is scheduled AND the name of that trigger is "nightly_build"
        when: pipeline.trigger.type == "schedule" and pipeline.schedule.name == "nightly_build"
        jobs:
          - build
          - deploy

Notes

  • Environment variables cannot be used in expressions and logic statements. You can only use pipeline values or pipeline parameters.

    Expressions and logic statements are evaluated at compilation time whereas the actual values of environment variables only become available at run time.

    See the "Can I use environment variables in conditional statements?" article for more information.

Run a workflow when open pull requests are modified

Goal

You may want to have a workflow run when open pull requests are updated with new commits (synchronize) or labeled.

Steps

  • Update your CircleCI config to use the pipeline.event.name and pipeline.event.action Pipeline Values.

    workflows:
    # The "run_on_pr" workflow only runs when open pull requests are labeled or updated with new commits.
      run_on_pr:
        when: (pipeline.event.name == "pull_request" and pipeline.event.action == "labeled") or (pipeline.event.name == "pull_request" and pipeline.event.action == "synchronize")

Notes

  • The pipeline.event.name and pipeline.event.action variables are not available for GitLab pipelines.

  • Environment variables cannot be used in expressions and logic statements. You can only use pipeline values or pipeline parameters.

    Expressions and logic statements are evaluated at compilation time whereas the actual values of environment variables only become available at run time.

    See the "Can I use environment variables in conditional statements?" article for more information.

Use no-op jobs to create a cleaner workflow graph

Goal

You may want to organize your workflow graph to make it more readable without adding additional build time or consuming credits. Using no-op jobs can help create a cleaner structure for complex workflows with many dependencies.

The following snippet shows how your workflow configuration might look without using a no-op job:

version: 2.1

jobs:
  build:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm ci
      - run: npm run build

  test-unit:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run test:unit

  test-integration:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run test:integration

  test-e2e:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run test:e2e

  deploy:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run deploy

workflows:
  build-test-deploy:
    jobs:
      - build
      - test-unit:
          requires:
            - build
      - test-integration:
          requires:
            - build
      - test-e2e:
          requires:
            - build
      - deploy:
          requires:
            - test-unit
            - test-integration
            - test-e2e

Notice how the deploy job needs to explicitly list all three test jobs in its requires section. As your workflow grows, this list could become longer and more difficult to maintain.

Here is the workflow graph for this config:

Workflow graph showing snippet without `no-op` job
Figure 3. Workflow graph before adding no-op job

Steps

  1. Add a no-op job to your CircleCI configuration file.

    jobs:
      test-coordinator:
        type: no-op
    
      test-unit:
        docker:
          - image: cimg/node:20.10
        steps:
          - checkout
          - run: npm run test:unit
    
      test-integration:
        docker:
          - image: cimg/node:20.10
        steps:
          - checkout
          - run: npm run test:integration
    
      test-e2e:
        docker:
          - image: cimg/node:20.10
        steps:
          - checkout
          - run: npm run test:e2e
  2. Configure your workflow to use the no-op job as a coordinator for downstream jobs.

    workflows:
      build-test-deploy:
        jobs:
          - build
          - test-unit:
              requires:
                - build
          - test-integration:
              requires:
                - build
          - test-e2e:
              requires:
                - build
          - test-coordinator:
              requires:
                - test-unit
                - test-integration
                - test-e2e
          - deploy:
              requires:
                - test-coordinator

The workflow graph will now look like this:

Workflow graph showing snippet with `no-op` job
Figure 4. Workflow graph after adding no-op job

Notes

  • The no-op job performs no actions and consumes no credits. This makes it an ideal structural element in your workflow.

  • This pattern is useful when you want to visually group related jobs in the CircleCI UI.

  • Using no-op jobs can simplify workflow management when dealing with complex dependency chains.

  • Without the test-coordinator job, the deploy job would need to list all test jobs in its requires section.

Simplify GitHub required status checks with no-op jobs

Goal

You may want to simplify the management of GitHub required status checks for your branch protection rules. Use a single no-op job as a gatekeeper that depends on all your critical checks. This approach means you only need to manage a single required status check rather than multiple status checks in GitHub.

The following snippet shows how your workflow configuration might look without using a no-op job:

version: 2.1

jobs:
  lint:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run lint

  test:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm test

  security-scan:
    docker:
      - image: cimg/node:20.10
    steps:
      - checkout
      - run: npm run security-check

workflows:
  main:
    jobs:
      - lint
      - test
      - security-scan

With this configuration, in GitHub you would need to set up branch protection rules for each job (lint, test, security-scan). You would need to update both CircleCI config and GitHub settings when adding more jobs.

Steps

  1. Add a no-op job to your CircleCI configuration file that will serve as your single required status check. In this example the no-op job is all-tests-passed:

    jobs:
      all-tests-passed:
        type: no-op
    
      lint:
        docker:
          - image: cimg/node:20.10
        steps:
          - checkout
          - run: npm run lint
    
      test:
        docker:
          - image: cimg/node:20.10
        steps:
          - checkout
          - run: npm test
    
      security-scan:
        docker:
          - image: cimg/node:20.10
        steps:
          - checkout
          - run: npm run security-check
  2. Configure your workflow to make all critical jobs prerequisites for the no-op job. The all-tests-passed will only succeed if all its required jobs succeed.

    workflows:
      main:
        jobs:
          - lint
          - test
          - security-scan
          - all-tests-passed:
              requires:
                - lint
                - test
                - security-scan
  3. In GitHub, set up a branch protection rule to require only the all-tests-passed job as a required status check. See the GitHub docs for more information on branch protection rules.

Notes

  • This approach simplifies branch protection management in GitHub by requiring only one status check.

  • When you add new checks, you only need to update the requires list for your no-op job.

  • The no-op job will succeed only if all its required jobs succeed.

  • This pattern helps keep GitHub and CircleCI configurations in sync without manual updates.