Set up multiple configuration files for a project
A CircleCI project integrated with GitHub can have one or more pipelines, each defined by a separate configuration file. CircleCI configuration files can be stored in the same repo as your code, or even in different repos if you wish.
Each pipeline can be configured to run on one or many specific trigger events.
If your existing configuration file is getting too large to manage, and it includes several unrelated workflows, consider splitting it into more than one file.
Using multiple pipelines and triggers, you can build different pipelines for different events, For example, you could have:
-
A
build-testpipeline that runs on every push. Select "All pushes". -
A
e2e-testspipeline, longer and more costly, that runs once when the PR is taken out of draft. Select "PR marked ready for review". -
A
teardown-envpipeline that runs when a PR is merged. Select "PR merged".
For a full list, see the GitHub trigger event options page.
Prerequisites
To use multiple pipelines and configs for a project in CircleCI, you will need to meet the following prerequisites:
-
A CircleCI account connected to your code in GitHub. You can sign up for free. Any of CircleCI’s GitHub integrations are supported, read our community forum for background on how this functionality can be used with an org integrated CircleCI’s GitHub OAuth App. To use multiple pipelines for a project you will use the CircleCI GitHub App integration.
-
A project set up in CircleCI that you want to configure with multiple, distinct pipelines defined with different configuration files. See the Create a project in CircleCI page for steps to get your project set up in CircleCI.
1. Add additional configuration files to your project repository
To use more than one configuration for your project you will create more than one pipeline. You can set up separate triggers for each pipeline. Each pipeline is defined by a YAML config file.
In this first step, create the configs you need to build your project. Config files can be stored anywhere you choose within your GitHub org. This can be in the same repo, or a different repo. Your configuration files can have any name you choose, but must have the .yml extension.
If your project is already set up in CircleCI, a configuration file will likely exist already. If you do not see a .circleci directory containing a YAML config, check which branch you are on. You may have set this up on a non-default branch during the set up process.
Each of your configuration files represents the full set of commands necessary for a pipeline to execute so it should be complete with all necessary elements, for example, version, jobs, workflows, commands, executors, orbs etc.
| Splitting up an existing configuration? If you are following these steps because you have an existing project building on CircleCI and you want to split your configuration into multiple files, you can create the config files you need and move over parts of your existing configuration file to your new YAML files. |
2. Set up a pipeline
Next, add a new pipeline for each config you have. Create and edit pipeline in your .
-
In the CircleCI web app, select your org from the org cards on your user homepage.
-
Select Projects from the sidebar and locate your project from the list. You can use the search to help.
-
Select the ellipsis () next to your project and select Project Settings.
-
Select Pipelines in the sidebar.
-
Select Add Pipeline.
-
Complete the required form fields:
-
Give your pipeline a descriptive name.
-
Ensure GitHub App is selected under Integration.
-
If you see a Connect button, select it and follow instructions to install the GitHub App for your org. You will need to be an org admin to install the GitHub App.
-
Select the same repository as the project you are building under Config Source. Specify where your config file is located under Config File Path.
-
Under Build Assets select your code source under Checkout Source — this is the code that will be checked out when using the
checkoutstep in your config file.
-
-
Select Save
3. Create a new trigger
Next, create a trigger for each pipeline you have created for your project. Alternatively, you can trigger a pipeline of your choosing manually via the web app.
-
Following from the previous step, select Triggers in the sidebar
-
Select Add Trigger.
-
Select GitHub App in the dropdown menu.
-
Select Next.
-
Complete the required form fields:
-
Give your trigger a descriptive name.
-
Select your project repository from the dropdown.
-
Choose your new pipeline from the "Choose config to run" menu.
-
Choose a trigger event options from the "Run on" menu.
Using multiple pipelines and triggers, you can build different pipelines for different events, For example, you could have:
-
A
build-testpipeline that runs on every push. Select "All pushes". -
A
e2e-testspipeline, longer and more costly, that runs once when the PR is taken out of draft. Select "PR marked ready for review". -
A
teardown-envpipeline that runs when a PR is merged. Select "PR merged".
For a full list, see the GitHub trigger event options page.
-
-
-
Select Save
Conclusion
You should now have more than one CircleCI config file for your project, each config connected to a separate pipeline, and each pipeline connected to a trigger, as required.