Open Listings logo

How Open Listings shipped a re-brand with CircleCI

Running 100,000+ tests a day with CircleCI

The Challenge Prior To CircleCI

At Open Listings, we’re a scrappy and passionate team focused on an ambitious mission: making homebuying simple and affordable. Our product is fairly complex, as it has to address homebuyers’ different needs through each distinct step of the homebuying process. There’s no practical way to support all these use cases without thorough automated testing.

Fortunately, we’ve been serious about automated testing from commit one. We run unit tests in Ruby that test our algorithms for things like dispatching agents to properties, along with how the application interacts with our MongoDB database; Ruby functional tests that test our API responses; end to end integration tests with capybara, phantomjs, and Selenium; and front end javascript tests with Jest that test our React.js code. Our tests run in a Docker container that is as close as possible to the environment we run in production on Amazon AWS.

While we already had a CI solution in place, as we added more developers, we experienced issues with scaling rapidly. As our engineering team went from 3 to 8 people in a few short months, having a CI solution that was more stable and could scale with our growth became a pressing need. So, our engineering team sought out a new solution and thankfully found CircleCI.

The Solution

Our most recent product challenge was a complete website overhaul, redesign and rebrand with the goal of helping home buyers clearly understand Open Listings’ all-in-one homebuying service.

CircleCI gives us confidence that we aren’t breaking existing functionality as we ship new code. With over a million unique visitors to our site every month, we couldn’t allow our redesign to disrupt continuity of service to our users.

Much of our website traffic comes from individual property pages, like the one seen below: Open Listing property page details

These pages include feature-rich modules that help homebuyers consider the costs of ownership, book upcoming house tours, and download property reports. Our business relies heavily on the stability of the code on these pages. Each one of these features has extensive end-to-end browser testing that ensures it functions all the way through the stack.

By their very nature, these tests can be time-consuming and error prone. CircleCI lets us run tests in parallel, reducing overall build time. Additionally, the maturity and stability of CircleCI’s system means we don’t have to spend time diagnosing failures that are not caused by our application.

How CircleCI Helped Our Engineering Team Grow & Thrive

Working on a redesign of this scale, we had multiple stakeholders and numerous commits coming from different authors. Using a collaborative tool like CircleCI during our site redesign cut down on at least an hour per day of meetings to reconcile work from different areas of our engineering team.

With our previous stack, each time we hired an engineer, we would have to schedule a phone call and sign an additional annual contract to increase capacity. But with CircleCI, there is a dial we can turn instead of picking up the phone. And now with their Performance Plan, the platform automatically scales to whatever capacity we are using—the dial turns itself!

Now that we’ve shipped our new site, we’ve begun the process of building an app in React Native. We started by building a simple webview version with native controls and are now building in new native features. CircleCI is awesome because it allows us to write tests using Mac hardware. This lets us test our mobile app using the same tools that everyone on the team is already familiar with.

CircleCI helps us focus on what really matters: shipping features for customers, instead of writing CI tools and wrangling in-house testing infrastructure. When you are running 100,000 tests per day and operating at that type of scale, eliminating unnecessary work is a major relief.

Because CircleCI integrates with Docker, we have the option of running Docker containers in development, which serves as a great line of defense against mistakes. If we can run Docker in both environments, then we can be confident we will not encounter new bugs when we deploy to production.

From an emotional and team-morale standpoint, we’ve noticed a difference in how well our engineering team members work together. Plus, our velocity in shipping new features has accelerated over time as a result of collaborating as one cohesive unit.

As a 3-year-old product with a large code base and a number of new developers on the team, CircleCI has been invaluable. Business requirements are documented in the test suite so we can be confident that even as we onboard new engineers quickly, we can maintain our great product experience.

In the end, our redesign went well and CircleCI helped us deploy without a hitch. If I were recommending CircleCI to another engineering team, I’d summarize its value by saying:

  • Platform stability & uptime
  • Ability to parallelize workflows can lead to dramatic reduction in test build time
  • Not having to run your own CI stack means no wasted engineering time working on non-core business problems
  • Docker makes it super easy to plug and play, especially if you’re already running Docker in production
  • Ability to run mobile app tests in native environments
  • Performance Plan lets you easily scale if you’re growing quickly (hopefully you are!)

Alex Farrill
“As a 3-year-old product with a large code base and a number of new developers on the team, CircleCI has been invaluable. Business requirements are documented in the test suite so we can be confident that even as we onboard new engineers quickly, we can maintain our great product experience.”

Alex Farrill
CTO at Open Listings


Thank You for Submitting Your Info

You should receive an automated response notifying you that we received your info. Someone from our Enterprise team will be reaching out to you shortly.

CircleCI Success Logo