How Open Listings shipped a re-brand with CircleCI
Running 100,000+ tests a day with CircleCI
The Challenge Prior To CircleCI
At Open Listings, we’re a scrappy and passionate team focused on an ambitious mission: making homebuying simple and affordable. Our product is fairly complex, as it has to address homebuyers’ different needs through each distinct step of the homebuying process. There’s no practical way to support all these use cases without thorough automated testing.
While we already had a CI solution in place, as we added more developers, we experienced issues with scaling rapidly. As our engineering team went from 3 to 8 people in a few short months, having a CI solution that was more stable and could scale with our growth became a pressing need. So, our engineering team sought out a new solution and thankfully found CircleCI.
Our most recent product challenge was a complete website overhaul, redesign and rebrand with the goal of helping home buyers clearly understand Open Listings’ all-in-one homebuying service.
CircleCI gives us confidence that we aren’t breaking existing functionality as we ship new code. With over a million unique visitors to our site every month, we couldn’t allow our redesign to disrupt continuity of service to our users.
Much of our website traffic comes from individual property pages, like the one seen below:
These pages include feature-rich modules that help homebuyers consider the costs of ownership, book upcoming house tours, and download property reports. Our business relies heavily on the stability of the code on these pages. Each one of these features has extensive end-to-end browser testing that ensures it functions all the way through the stack.
By their very nature, these tests can be time-consuming and error prone. CircleCI lets us run tests in parallel, reducing overall build time. Additionally, the maturity and stability of CircleCI’s system means we don’t have to spend time diagnosing failures that are not caused by our application.
How CircleCI Helped Our Engineering Team Grow & Thrive
Working on a redesign of this scale, we had multiple stakeholders and numerous commits coming from different authors. Using a collaborative tool like CircleCI during our site redesign cut down on at least an hour per day of meetings to reconcile work from different areas of our engineering team.
With our previous stack, each time we hired an engineer, we would have to schedule a phone call and sign an additional annual contract to increase capacity. But with CircleCI, there is a dial we can turn instead of picking up the phone. And now with their Performance Plan, the platform automatically scales to whatever capacity we are using—the dial turns itself!
Now that we’ve shipped our new site, we’ve begun the process of building an app in React Native. We started by building a simple webview version with native controls and are now building in new native features. CircleCI is awesome because it allows us to write tests using Mac hardware. This lets us test our mobile app using the same tools that everyone on the team is already familiar with.
CircleCI helps us focus on what really matters: shipping features for customers, instead of writing CI tools and wrangling in-house testing infrastructure. When you are running 100,000 tests per day and operating at that type of scale, eliminating unnecessary work is a major relief.
Because CircleCI integrates with Docker, we have the option of running Docker containers in development, which serves as a great line of defense against mistakes. If we can run Docker in both environments, then we can be confident we will not encounter new bugs when we deploy to production.
From an emotional and team-morale standpoint, we’ve noticed a difference in how well our engineering team members work together. Plus, our velocity in shipping new features has accelerated over time as a result of collaborating as one cohesive unit.
As a 3-year-old product with a large code base and a number of new developers on the team, CircleCI has been invaluable. Business requirements are documented in the test suite so we can be confident that even as we onboard new engineers quickly, we can maintain our great product experience.
In the end, our redesign went well and CircleCI helped us deploy without a hitch. If I were recommending CircleCI to another engineering team, I’d summarize its value by saying:
- Platform stability & uptime
- Ability to parallelize workflows can lead to dramatic reduction in test build time
- Not having to run your own CI stack means no wasted engineering time working on non-core business problems
- Docker makes it super easy to plug and play, especially if you’re already running Docker in production
- Ability to run mobile app tests in native environments
- Performance Plan lets you easily scale if you’re growing quickly (hopefully you are!)
“As a 3-year-old product with a large code base and a number of new developers on the team, CircleCI has been invaluable. Business requirements are documented in the test suite so we can be confident that even as we onboard new engineers quickly, we can maintain our great product experience. ”
CTO at Open Listings
Summer of 2017