Scalable Continuous Integration with CircleCI

“We were able to rapidly grow our team and code base without fear of outgrowing CircleCI.”
John Duff, Director of Engineering, Shopify

Developers

130

Funding

122m

Technology

Ruby

Past Setup

Internal

Background

Shopify has a simple goal: to make commerce better. They do this by making it easy for over 100,000 online and brick and mortar retailers to accept payments. They've experienced tremendous growth over the last several years and in order to serve their growing customer base they've had to double their engineering team from 60 to 130 in the last year alone. In addition to the usual growth challenges, they faced the problem of maintaining their test-driven and continuous deployment culture while keeping productivity high.

Shopify has always taken Continuous Integration (CI) and Continuous Deployment (CD) seriously and built an in-house tool early on to make both practices an integrated part of their workflow. As the team grew, this in-house tool required more and more work to maintain and the self-hosted server was struggling to handle the load.

Eventually the Shopify team was dedicating the equivalent of two full-time engineers to maintaining their CI tool, and even then it was not performing up to their needs. Custom configuration was required to add a new repository, it was difficult to integrate with other tools, and the test suite took too long. In the spring of 2013 freeing up developers to focus on the core product, creating a more streamlined and flexible developer workflow, and getting new products to market quickly were all high priorities for John Duff, Shopify's Director of Engineering. Much of this, he decided, could be accomplished with a better CI solution.

Data Analysis

In looking for a CI solution, Shopify needed something that was dependable, affordable, and would meet three core product criteria.

Several of the developers at Shopify had used CircleCI before and recommended it as potentially meeting all of these needs. In order to try out CircleCI several developers signed up for a 14-day free trial using their GitHub accounts and followed a couple of their Shopify repositories from within the CircleCI app. CircleCI then inferred their test environment based on existing code and their tests began to run automatically. The developers were able to get quick clarification on a few setup details via CircleCI's live chat support room, and within a few minutes they were convinced that CircleCI would meet and surpass their needs.

After using the product for a few days there were several features that set CircleCI apart from the others; easy scalability of both build containers and parallelism, a well documented REST API, and extensive customization and configurability options including SSH access to the build machines. This, combined with the easy setup and helpful support, convinced the team that CircleCI was the perfect solution.

Powerful Scalability

Shopify was undergoing rapid growth and they needed a solution that they would not outgrow.

Robust API

Shopify required a robust API that would allow integration into their existing workflow and tools.

Easy Customization

Shopify needed the ability to easily configure, customize, and debug build their machines.

Shopify
One of my favorite things about CircleCI is that their team really cares about making sure their customers get maximum value out of their product.
Arthur Neves
Developer at Shopify

Implementation

CircleCI integrates natively with GitHub, which Shopify was already using, so set up time was minimal; it only took a few minutes to follow the rest of their repos on CircleCI and to invite the rest of their team members. Once their tests were running, they started optimizing their containers and parallelization from within the CircleCI app so that their test suite would run as quickly as possible. Once they had the tests running for all their projects, the next step was setting up Continuous Deployment.

CD has always been a core part of the engineering culture at Shopify, so getting deployment set up with CircleCI was essential. To streamline their CD process, Shopify used the CircleCI API to build a custom 'Ship It' tool that allows any developer to deploy with the press of a button, as long as they have a green build on CircleCI. All they had to do to build this was verify that the pull request in question returned "outcome" : "success" from the CircleCI API after merging with master, and then allow the developer to deploy.

This same functionality can also be accomplished without using the API by putting the deployment script directly into the circle.yml file.

Shopify
CircleCI lets us be more agile and ship product faster. We can focus on delivering value to our customer, not maintaining CI infrastructure.
John Duff
Director of Engineering at Shopify

Results

Today, 1 year after initially switching to CircleCI, Shopify has scaled their engineering team to 130 team members who on average merge 300 pull requests and deploy 100 times per week. Thanks to CircleCI, they've managed to maintain their agile and efficient development process, with new projects being added effortlessly and everyone working off of a master branch (rather than having to maintain production and development branches). Their test suite runs faster than it ever did with their previous solution, and now that developers don't have to run tests on their local machine they can work on other projects while CircleCI runs their tests in the background. Shopify also uses CircleCI along with Capistrano to continuously deploy their application for anything from a small bug fix, to a package upgrade, to a new feature.

The Shopify team no longer has to worry about scaling their testing infrastructure, maintaining their test stack, or monitoring their deployments. They focus on building products that bring value to their customers while relying on CircleCI to ensure that they are able to get those products to market quickly and reliably.