With the new release of dynamic config via setup workflows, CircleCI customers can now use jobs and workflows, not only to execute work but to determine the work they want to run.

We built dynamic config because we know our users want more dynamism in the CircleCI build process. Historically, our platform has been very deterministic: the config is pre-set in a file based on the revision for a given pipeline. However, this determinism results in less flexibility for users to run what they want, when they want.

In the past, customers have tried to work around the static nature of CircleCI config by using scripts in pre-commit hooks to generate config before commit, or jobs to trigger pipeline runs via the API so they can set pipeline parameters. While these solutions work, they frequently involve strange workarounds or edge cases that are not easily solved.

How we enabled dynamic config

Realizing that jobs and workflows were one of the best ways to make our config more dynamic, we created a special workflow, known as a “setup workflow.” This workflow can contain jobs that “set up” a pipeline by computing pipeline parameters or generating YAML config. These parameters and config can then be passed into a CircleCI command/job at the end of the setup workflow, and any workflows or jobs defined in that config are added to the same pipeline.

One of the most exciting parts of this functionality is that because the setup workflow is normal CircleCI config, you can use orbs to incorporate common types of logic. We’re currently working on several orbs for this use case. For example, we’ve created a path filtering orb that runs git commands to analyze which files have changed in a commit and use regular expressions to set pipeline parameters that can be used to filter workflows or steps. Read our docs to learn more about the specific capabilities of dynamic config.

What’s next

We’re looking forward to using dynamic config and orbs as a way to help customers solve different use cases. A few of the ideas we’re considering:

  • An orb that would allow customers to combine configs, making it possible to break up large config files
  • Orbs that interpret the output of build tools like Bazel or Lerna
  • An orb that enables you to run the .circleci/config.yml for a particular branch (like your default branch), regardless of the branch or PR that triggered the pipeline.
  • An orb defining an SDK so that you can write your config using TypeScript

Head over to our community forum, Discuss, to let us know how you’re using dynamic config, and check out this tutorial to learn how to incorporate dynamic config into your CI/CD pipelines.