This page summarizes concurrency, and suggests ways to use concurrency to your advantage.
In CircleCI, concurrency refers to utilizing multiple containers to run multiple jobs at the same time. This is different from CircleCI parallelism, which is test-splitting across multiple containers. If you would like information on parallelism, visit the Running Tests in Parallel page.
Every plan with CircleCI includes the ability to run jobs concurrently. However, the number of jobs you can run concurrently differs between plans. Please refer to the pricing page for up-to-date information on the number of concurrent job runs included in your plan.
Within CircleCI, concurrent jobs can be classified as one of two situations:
Multiple running jobs configured in a single workflow
Multiple workflows running by multiple members within the organization
This page discusses the first point, and will provide examples of how to configure concurrent jobs in a single workflow. Configuring concurrency in a workflow allows multiple jobs to be run at the same time, in a single workflow, through the use of multiple containers.
To keep the system stable for all CircleCI customers, we implement different soft concurrency limits on each of the resource classes for different executors.
If jobs are queued, it is possible you are hitting these limits. Hitting your limit may depend on a few factors, such as how many people in your organization are running jobs at once, or if you have overlapping workflows.
Customers on a Performance or Scale plan can request an increase to those limits at no extra charge by contacting CircleCI support.
Concurrency in workflows
To run a set of concurrent jobs, you will need to add a
workflows section to your existing
The simple example below shows the default workflow orchestration with two concurrent jobs. The
workflows key needs to have a unique name. In this example, the unique name is
jobs key is nested under the uniquely named workflow, and contains the list of job names. Since the jobs have no dependencies, they will run concurrently.
version: 2.1 jobs: build: docker: - image: cimg/<language>:<version TAG> auth: username: mydockerhub-user password: $DOCKERHUB_PASSWORD # context / project UI env-var reference steps: - checkout - run: <command> test: docker: - image: cimg/<language>:<version TAG> auth: username: mydockerhub-user password: $DOCKERHUB_PASSWORD # context / project UI env-var reference steps: - checkout - run: <command> workflows: build_and_test: jobs: - build - test
Please note, if you are still using Server 2 and
version: 2, your
workflows section will need to specify the version again, as in the below snippet:
... workflows: version: 2 build_and_test: jobs: - build - test ...
Fan-out/fan-in workflow example
A more complex example of using concurrent workflows is running a common build job, fanning-out to run a set of acceptance test jobs concurrently, and then fanning-in to run a common deploy job. This flow is illustrated below.
.circleci/config.yml snippet below is an example of a
workflows section configured for fan-out/fan-in job execution. In this example, as soon as the
build job finishes successfully, all four acceptance test
jobs start. The
deploy job must wait for all four acceptance test jobs to complete successfully before it starts.
... workflows: build_accept_deploy: jobs: - build - acceptance_test_1: requires: - build - acceptance_test_2: requires: - build - acceptance_test_3: requires: - build - acceptance_test_4: requires: - build - deploy: requires: - acceptance_test_1 - acceptance_test_2 - acceptance_test_3 - acceptance_test_4 ...
Please note, if you are still using CircleCI Server 2 with
version: 2, you will need to specify the version in the
workflows section, as mentioned in the Concurrency in workflows section.
Help make this document better
This guide, as well as the rest of our docs, are open source and available on GitHub. We welcome your contributions.
- Suggest an edit to this page (please read the contributing guide first).
- To report a problem in the documentation, or to submit feedback and comments, please open an issue on GitHub.
- CircleCI is always seeking ways to improve your experience with our platform. If you would like to share feedback, please join our research community.
Our support engineers are available to help with service issues, billing, or account related questions, and can help troubleshoot build configurations. Contact our support engineers by opening a ticket.
You can also visit our support site to find support articles, community forums, and training resources.
CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.