This page will guide you through everything there is to know about continuous integration. We’ll cover the nuanced differences between continuous integration, continuous deployment, and continuous delivery, explaining use-cases and best-practices along the way to automating the building and testing of your software.

What is continuous integration?

Continuous integration (CI) is a software development strategy that increases the speed of development while ensuring the quality of the code that teams deploy. Developers continually commit code in small increments (at least daily, or even several times a day), which is then automatically built and tested before it is merged with the shared repository.

What is continuous integration?
0:54

Continuous integration: an overview

  • Every developer commits daily, or even more often, to a shared mainline code repository
  • Every commit triggers an automated build and test of the codebase
  • If the build or any test fails, it’s repaired quickly – often within minutes

Continuous integration works hand-in-hand with Agile methodologies. Team members work on incremental “stories” and the code for these software changes is merged incrementally into the shared software repository multiple times a day.

Continuous integration can be used for many types of software projects. For example, we use continuous integration for updating the CircleCI website:

  1. Developer or content publisher creates a new branch of code in GitHub, updates the HTML on the page, and then commits the changes.

  2. When the developer has completed all the changes on the particular branch, they then create a pull request on GitHub.

  3. GitHub is integrated with CircleCI. CircleCI runs the build processes on the code and then runs the automated test scripts.

  4. If CircleCI encounters any errors (build status red), the developer is notified via email.

  5. Otherwise, the user is notified that the build was successful (green) and that the code has been pushed to the staging server. This allows the developer to preview the web page.

  6. Once verified, the developer can then merge the new branch of code into production.

What is the difference between continuous integration, continuous delivery, and continuous deployment?

What is the difference between continuous integration, delivery, and deployment?
0:56

Continuous integration

The automated building and testing of your application on every new commit.

Continuous delivery

A state where your application is always ready to be deployed. A manual step is required to actually deploy the application.

Continuous deployment

The automation of building, testing, and deploying. If all tests pass, every new commit will push new code through the entire development pipeline to production with no manual intervention.

Illustration of the difference between continuous delivery and continuous deployment. The main difference is that continuous delivery requires manual intervention to deploy. Continuous deployment is completely automated.

Continuous Delivery By Martin Fowler
Continuous Delivery vs Continuous Deployment By Jez Humble
Continuous Deployment By Timothy Fitz
Continuous Delivery By Wikipedia
A brief history of DevOps, Part III: Automated Testing and Continuous Integration By Alek Sharma
A Brief History of DevOps, Part IV: Continuous Delivery and Continuous Deployment By Alek Sharma

Continuous integration (CI) vs continuous deployment (CD)

Continuous integration automates the building and testing of your software. Continuous deployment is an extension of this automation and allows for your software to be deployed after every code commit that passes your test suite. The most successful development teams deploy their software often.

The software development process consists of three main centers of work:

  1. Creation of the code: This is where the software development team is the most creative. They are turning user stories into elegant code.

  2. Orchestration of new code as software: This is where the heart of continuous integration lives. Once the developer commits code, the continuous integration build server automates the build, test, and deploy process.

  3. Logistics: This is the final step of shipping the new software to production. Tools in this part of the process monitor and manage the software in production.

The most important tool for the creation process is the repository used to hold the code. CircleCI integrates with GitHub and Bitbucket. Our server solution also integrates with GitHub Enterprise.

CircleCI lives in the orchestration center of the work. It does the heavy lifting of turning the code into working software. Users configure how the code is processed via a YAML file. The YAML file allows configuration via code and is almost infinitely configurable, allowing teams to build and test their way.

The logistics part of the software development process includes getting the tested and approved code into production. This includes the setup, configuration, and management of the production servers. To gain the full benefit of continuous integration, continuous deployment is necessary.

How to get started with CI/CD

It is important to get a continuous integration (CI) pipeline set up as early as possible in the development of your application. Once tests are added to your CI pipeline, you can continue to innovate and build with confidence knowing that you cannot merge code to main that doesn’t pass your tests. Additionally, developing a continuous deployment workflow into your CI pipeline provides the automation of deployment. With CircleCI, you can automate your software’s build, test, and deployment and your engineers can get back to work building the features that your users care about the most.

Getting started with continuous integration
1:58

Getting started

  1. Make sure everyone is on the same page with respect to what you’re trying to achieve by introducing a new CI/CD tool.

  2. Always start small. Don’t try to make your entire organization into a model DevOps team overnight. Instead, make process changes on one team to see if they work within your organization. If you see improvements, keep moving incrementally.

  3. Do what works for you. Know that DevOps might not be the right solution for your organization. Some companies have been successful for a long time without DevOps, and it might not be right for them, given their culture or their product needs. For example, if confidentiality is a big part of your company’s product strategy, then shipping incrementally to get feedback would not work for you, as you’d need to keep all product details under lock and key until a big launch. In that environment, it would be very difficult to build a DevOps culture.

  4. Always measure. Before you start any improvement plan, get accurate metrics for where you’re currently at (i.e. our dev cycle takes X amount of time). Do this before you’ve invited a site reliability engineer onto the development team. Then after a small amount of time, you’ll be able to see whether it’s been effective. As an example: When a lot of Agile transformation was happening, many companies adopted standups, without really understanding why, or measuring whether it had a positive effect on their team. This likely wasted more time than it saved.

  5. Do not try to automate everything. At least not all at once. One misconception about DevOps is that all the infrastructure provisioning and the configuration management must be done automatically. This is referred to as “infrastructure as code.” But some things work better when they are manual; automation is not the solution for everything. Sometimes, you have to start the manual way to figure out what the best solution for automation actually is.

Continuous integration is not just about implementing a new tool. It is about changing how the development team works. It’s a new mindset. And, making culture changes in an organization is not easy. The best way to get started with continuous integration is to start small.

Use your next small project as a proof of concept and get the following fundamentals into place:

  • A rigorous testing practice, ideally using automated tests.
  • Consistent software environments, from testing to production.
  • Training on continuous integration practices.
  • Reports to measure key metrics.

Once your team starts committing regularly and in small increments, they’ll see how much easier it is to be responsive to bugs. And, as they resolve bugs faster, they will deliver features even faster. This will give your team the momentum to scale continuous integration across the organization.

What is the best continuous integration and continuous deployment (CI/CD) tool?

DevOps is a very specific collaboration between developers and operations teams. In essence, it indicates that you’ve culturally adopted development practices into your infrastructure, and operational practices into your development cycle. What does this look like in practice? It can mean maintaining infrastructure as code, or creating immutable infrastructure by building reusable components so you can tear down or up whenever you want, giving you version control and a history of changes you’ve made. It also means getting all product contributors to care more about the end result of what they’re working on—how does it function in the world? How are users interacting with it? Getting people to truly care about quality means caring about both business value and usability. That’s the result of true DevOps adoption.

This kind of widespread buy-in is particularly difficult for software teams to achieve because it requires a lot of cooperation from people with different skills and domain expertise. Pulling this off requires both cross-functional team structure and thoughtful communication. For example, if an engineer needs to talk to someone on the business side about a database problem, they need to not just show the data they’re working from, but give necessary context and focus that person’s attention on what they should care about and why.

So, should you bring in that new tool? The answer is: maybe. Tools can sometimes seem like a quick fix, but they are not a one-size-fits-all solution. Keeping these considerations in mind before you bring in a new DevOps tool will increase your chances of success. The best CI/CD tool is the one that works for your team.

How to bring a new tool to your team (and not get squashed in the process) By Emma Webb

Hosted vs on-premise continuous integration

We can host continuous integration for you on our cloud-managed option (SaaS) or you can run CircleCI behind a firewall on your private infrastructure.

Cloud

We oversee the setup, security, and maintenance of your continuous integration instance(s), and we provide immediate access to feature releases and automatic upgrades alleviating the need for maintenance. Authorize with GitHub or Bitbucket and go from sign up to green build in minutes.

Server

Install CircleCI on a private server that your team sets up and maintains for security. This provides full System Administrator control for complete customization. Your team determines the update cadence to fit your maintenance schedule.

Benefits of continuous integration

The fundamental tenet of continuous integration is quite simple: commit and integrate your code often—daily at a minimum. Such a small change in your software development process can yield big results.

Benefits of continuous integration
1:31

Using this development strategy, you can:

  • Improve team productivity and efficiency
  • Accelerate speed to market
  • Identify product/market fit
  • Release higher quality, more stable products
  • Increase customer satisfaction
  • Keep developers happy and shipping code

How do these benefits accrue from such a simple change in your workflow?

When you commit more often, you can identify and resolve merge conflicts earlier, or avoid them altogether. So, instead of writing a thousand lines of code and finding an error, you’ve only written a hundred. Finding the bugs in your code becomes an errand of a few minutes instead of a few hours. This leads to improved team productivity and helps developers ship working code more quickly.

Shipping new features quickly means increasing your speed to market. This gives your team a competitive edge in two key ways:

  1. Your customers are getting access to new features faster, leading to higher customer satisfaction

  2. Your company is getting a faster return on investment from new features. Rather than waiting for the next milestone to release code, you can deliver value as soon as a new feature is ready for the market.

CI/CD for open source projects

CircleCI offers every organization on our free plan 400,000 credits per month to use for open-source Linux builds. To access this, just keep your repo public. If you’re building on macOS, CircleCI also offers organizations on our free plan 25,000 free credits per month to use on open source macOS builds. To get access, contact us at billing@circleci.com.

Continuous integration best practices

Developers who practice continuous integration commit early and often. This allows them to detect conflicts before deploying code to production. And, having a smaller commit makes it easier to troubleshoot the code if there are any issues. Committing software daily, or even more often, is necessary for continuous integration, but not sufficient.

Best practices of continuous integration
1:43

To use continuous integration successfully, the team must:

  • Make testing an integral part of the development process. Tests should be written as code is created.
  • Ensure that the testing environment mirrors production.
  • Use coding best practices, such as pair programming.
  • Automate the deploy workflow.

Fostering a rigorous testing culture is the most important element that a company needs for successful continuous integration. In order to confidently integrate new code into the mainline, the team needs the confidence that the code is sound. Engineers should write tests as each feature is being developed. At CircleCI, we run tests on any new code that is committed and our system alerts the developer if they have a successful “green” build, or if they need to fix issues because the build was “red.” Without tests, of course, green builds are meaningless. Ideally, the team is using automated testing, though that is not a requirement. QA services, such as Rainforest QA, can be integrated into the continuous integration process.

To support your rigorous testing culture, it’s important that the testing environment mirrors the production environment. Otherwise, you have no guarantee that what you’re testing will work in production. This means the testing environment should use the same version of the database, web server configuration, artifacts, etc.

Another best practice for software development is pairing during coding. For more complex pieces of functionality, pairs discuss the architecture approach before a single line of code is written. Before any code is merged into production, another developer always reviews the code. This helps ensure that coding best practices have been used, the code does not conflict with existing code or code that another developer is working on, and the new functionality is scalable.

Finally, to ensure that the entire software development pipeline is fast and efficient, the deploy workflow should also be automated. By automating the deploy workflow, the team gets their finished code to production more quickly. Because, after all, what’s the point of developing software quickly if it’s not getting to the customer?

Key metrics for tracking continuous integration

Continuous integration is a worthy goal of any software development team. But, how do you know if you’ve achieved your goal? This is where key performance indicators help. You need to measure the processes that make up the software development lifecycle in order to make improvements to it. This is how you will know if the changes to your process are making a difference.

How to measure the impact of continuous integration
2:17

Here are 4 key performance indicators (KPIs) to measure your software development:

  • Lead time: Lead time is the length of time that it takes for your workflow to run from first trigger to completion. It is a measure of how quickly you can get a signal. A short lead time requires a maximally automated software development pipeline. That is why teams use continuous integration to automate building and testing. Automation of the pipeline via CI shrinks deployment timelines of weeks to months down to hours, if not minutes.
  • Deployment frequency: How often your team kicks off a workflow is an indication of how many discrete units of work are moving through the development pipeline. With a fully automated software development pipeline, even complicated updates to your codebase can be deployed automatically. Hotfixes are available immediately — and they go through the same rigorous testing as any other update.
  • Mean time to recovery (MTTR): Mean time to recovery is contingent on getting actionable information, fast. Getting a failed status returned quickly will enable you to get from red to green in the shortest amount of time. Adopting CI/CD practices enables rapid feedback loops, and is the best way to ensure fast signal for your developers. A robust CI/CD practice includes realtime artifacts such as logs and coverage reports from your test suite and gives devs the ability to troubleshoot in an environment equivalent to the production environment.
  • Change fail percentage: The highest performing teams rarely push bad code to their default branch. But this doesn’t mean that these teams never write faulty code. Testing and security checks are performed on a separate branch and only when everything passes is a merge allowed to take place. A team should know that their code works well before it is merged into the default branch.

It’s helpful to have a regular cadence of reporting. Automated daily reports help ensure that if there is an error, the team catches it fast. And, analyzing the weekly reports allows teams to dive much deeper into the CI processes and find improvement areas.

Download The 2023 State of Software Delivery

Adding tests to your CI/CD pipeline

Let’s talk about the different types of tests:

Unit/component test

These cover the smallest possible component, unit, or functionality. They’re the cheapest and fastest tests to run since they don’t require a lot of dependencies or mocking. These should be done early to get them out of the way.

Integration test

These check how well each unit from the previous stage works with the other components, units, and functionalities. In a broader sense, it can test how services (such as APIs) integrate with one another.

UI layer testing

This is automated browser-based testing which tests basic user flow. It is expensive to set up and slow to run, so it should happen later in the pipeline.

Now let’s talk about how these tests fit into a software development pipeline. Each test has a specific role and place.

Local environment testing

This environment is private, limited to a single developer and their laptop. This is the easiest environment in which to make changes and test your own implementation.

The “local” we are talking about here is really just a “personal environment.” It could be on your laptop, but you could just as easily use a cloud environment if the application is too big to run on your local laptop. The key here is it’s a smaller-scale instance, that’s yours alone, in which you can test and debug your implementation while you’re developing, without interrupting other developers on your team. Because nothing in your local environment is visible to others, your team can agree to integrate a Git pre-push hook into your repo to ensure the local environment is used, and run automated tests before code gets pushed to a remote, or shared repository.

Tests

The tests I recommend before moving out of a local environment are unit tests, integration testing with mocked components, and UI testing to the degree that it’s possible. The more tests you can do in this environment, the more room you’ll have to be successful in an integration environment.

Scope of responsibility

The scope here covers just the implementation or functionality of what you’re building. Does your component work by itself? The surface area is relatively small. But, this is the last environment before you merge to a shared environment. It’s important to test responsibly, even in your local environment, to ensure that you don’t break the shared environment later on.

CI environment tests

This is the shortest-lived environment; it lives within the build. It gets created when a build gets triggered and torn down once the build is done. It’s also the most unstable environment. Since other developers could be deploying at the same time, the CI environment has a lot of deployment activities that are happening concurrently. As a result, the CI can, and often does, break. That’s ok–it’s meant to break. The key is fixing it when it does.

If you were unable to spin up your entire application in a local environment (which is highly likely), the CI environment will be the first environment in which you’ll be able to do browser-driven testing. You’ll also be able to do any UI testing that you weren’t able to do in your local environment.

In this environment, you should be using mock external services and databases to keep tests running fast.

The recommended automation of the lifetime of the CI environment is this: as soon as you merge the code, the CI environment automatically spins up, runs that code, tells you whether it’s safe or not, and then tears itself down. Using Docker to automatically spin up the environment will save time, and the whole process of automated builds and environment creation makes it easier for team members to commit more often.

The maximum benefit you can get the CI environment is based on the speed at which you can spin it up, because that will determine the length of your feedback loop. If your entire application takes something like 30 minutes to spin up and deploy, you may want to consider doing a partial deployment instead. Since the CI environment spins up and down quickly, you should mock most of your external services. Keep in mind you’re only testing the integrations of the core components rather than any external services.

Tests

Unit tests, integration testing with mocked components, and UI testing with mocked data.

Scope of responsibility

Here you are concerned with the shared codebase you’re merging your code into. Does your component work with the rest of your service?

Development environment tests

Everyone in the organization has visibility into this environment; any developer can log in and run it as an application. This environment can be used by all the developers for testing and debugging.

The development environment is a shared environment with other developers. In this environment, every service within the application is getting deployed. These environments are very unstable because there are constant changes from different teams all the time. It’s important to note here that your integration and browser-based tests may now fail, even if they passed in the CI environment. That’s because they are now fully integrated with outside services and other services that are also in development.

Whereas the CI environment is just for your team (or for one service of your product) and can be torn down between builds, this development environment is for your entire product codebase. If you choose to merge into a development environment for just your team, that will have a smaller scope of impact than merging into the full development environment shared by all teams. In the development environment, system health check monitoring is now required since it’s a fast-moving environment with many different components. Since it sits in the middle of the path to production, when this environment breaks, it has a detrimental effect on the environments that follow - a failure here blocks all changes and no code can move forward if the development environment breaks.

In the development environment, some external services are mocked and some are not, depending on how crucial each service is to what you’re testing, as well as the cost of connecting to that external service. If you are using mocked data in your development environment , make sure that a decent amount of test data is available.

Tests

Unit tests, integration testing with mocked components, UI testing with mocked data and system health checks.

Scope of responsibility

The scope increases to integrations with other services. Does your service work with the other services it will need to connect to?

QA environment tests

This is the first manually deployed environment. It is manually deployed because the QA team needs to decide which features are worth testing on their own, based on the structure of the changes. They might take a stacked change (Change A, B, and C) and test them each separately. In this scenario, they test Change A, and once they have assurance that it’s working as expected, they can move on and test Change B and when it throws an error, they know that the issue is isolated to Change B. Otherwise, in the case of just testing Change C, if it were to throw an error, it would block progress on C, B, and A.

The QA environment is a controlled and integrated environment. Here, the QA team is controlling what change is coming in. In contrast, in the development environment, any change can happen at any time. The build is now integrated with the services it will interact with in the application.

In this environment, we now have the same infrastructure and application as in production. We are using a representative subset of production data–close enough to production data to test. The QA engineers or testers understand what they should focus their tests on. Using manual deployment at this stage can help ensure that small changes are tested in an isolated environment to help identify bugs.

If you don’t use production or production-like data in your testing, QA testing can miss a lot. Use real external services if possible, so that QA can catch the real problems that would occur. The closer the QA environment can get to production, the more you will be able to trust your tests. As always, you have to weigh this assurance with the cost. Depending on your application’s functionality, you can still mock some of the external services with no ill effects.

Tests

This phase is an appropriate time for open-ended exploratory testing, as well as security testing. At this stage, you are hoping to catch not just technical problems but also potential user flow problems.

Staging environment tests

This is the last environment before production. The purpose of staging is to test your code in an environment that is almost identical to production. When you deploy something into staging, and it works, you can be reasonably assured that that version won’t fail in production and cause an outage. All environments help you catch potential issues; staging is the final check of confidence. Here it is important to have almost the same amount of data as you would in production. This enables you to do load testing, and test the scalability of the application in production.

Production-ready code should be deployed to this environment; again, almost the same as in production. Infrastructure, databases, and external service integration should be exactly the same as in production. It will be expensive to maintain and build– the only thing more expensive is not doing it, and breaking production. The scale can be smaller, but your setup and your configuration has to be the same.

This is the last gate before production to test the implementation, migration, configuration, and business requirements. It is strongly encouraged that you get business signoff before you get to staging. Otherwise, it will be very expensive to make a change. While you’re doing development, ensure that whoever owns the product or requested feature thoroughly understands what it is you’re building. Be on the same page the whole way through.

Tests

This environment can be used to do the most heavy-duty of all your testing so far: system testing, load testing, performance testing, and security testing.

Given the heft of the environment, it will be most beneficial if other core application functionality and internal integration testing has been done in previous environments.