An AI-powered MarTech leader boosts their build success rate 5X

• Problem: Iterable is the AI-powered customer communication platform that helps today’s top brands activate customers with joyful interactions at scale. Serving 1200+ customers in over 50 countries, their tool is a favorite of data-driven companies such as Calm, Redfin, and Box. Tasked with delivering against an ambitious roadmap, their engineering team struggled with flaky tests, leading to lower confidence in releasing features to production, and excess time spent rerunning tests.
• Solution: Iterable formed a new developer productivity team which leveraged CircleCI to instill more ownership of their software testing among dev teams. By implementing new testing standards, they created a culture of cross-team collaboration and code ownership – which led to an increase in build success rate by 530% in just six months.

As a rapidly-scaling SaaS marketing platform, Iterable plays in a highly competitive market. Staying on top means rapid product development (the company launched 37 new features in 2023 alone) while keeping customers happy (maintaining an impressive 99% customer satisfaction rate).

In the fast-moving SaaS space, innovation and reliability are twin aims: both vital, and often at odds with each other. Iterable needs to keep delivering new value to customers, while keeping their production code robust and performant.

Code quality as culture

When Mark Palestine, Sr. Engineering Manager for Platform, took over the Developer Productivity team at Iterable, he saw that while their code was being tested and they had good test coverage, the team lacked clear visibility into test issues once surfaced. While the team was testing locally and in a staging environment, issues with flaky testing made discovering potential code change issues more challenging. This led to a lack of confidence among the product engineers and a significant increase in time spent testing relative to the testing confidence output. Since psychological confidence is a cornerstone of good development practices, Palestine was highly motivated to refine the team’s testing process.

“Iterable is very conscientious about delivering a quality solution for customers,” says Palestine. “And when I took this team over, as a former developer, I could not imagine delivering code where I didn’t know if it’s going to work in production, or having to take extraordinary measures and time to ensure proper testing .”

Palestine knew that CircleCI could help his developers deliver better quality code to align with Iterable’s ambitious feature delivery roadmap. So he set to work leveraging CircleCI to instill more ownership of their software testing among the teams, creating alerts that kept things moving and increased their overall build success rate by over 5X. “I want the ability to discern [production-ready code]. Removing that anxiety, I’m creating a safe work environment for people to fail safely. We cant expect perfection, but we try to make things fail outside of production, then fix them, then send a better solution to production.”

Productivity as a service

The Dev Productivity team Palestine manages is made up of people who had previously worked on product engineering teams, “so they knew the struggles of having to spend extra hours rerunning tests that for all intents and purposes should have worked.”

While they don’t release features directly to customers, they have a kindred empathy for their own customers—the rest of the engineering org—and knew firsthand the value of using CircleCI to ship faster and more confidently. “There’s always the pressure on the developer to do more faster. On a regular basis we say that our team’s goal is to make our developers’ lives better. If we can do that, we’re supporting them in their mission to deliver quality features to customers with speed.”

Cross-collaboration generates more new features, faster

For Palestine, a 100% build success rate remains the target. Empowering his DevOps Productivity team to reach for this standard benefits more than just internal engineering metrics. The increased productivity empowers the business to ship more and help their customers do amazing things using their technology. Leading meditation app Calm used Iterable’s innovative features and robust and reliable tooling to 4X their revenue from new members.

Confident developers make happy customers

With CircleCI as a cornerstone of their code quality and platform reliability, Iterable is poised to keep rapidly scaling and grabbing marketshare. “We can ship code to production with more confidence,” he says. “We recover from incidents, whether they’re change-related or not, faster, because those events are going through our CI pipeline as well. Now we can distinguish the signal from the noise.”

If moving fast is important to your business, it’s vital that you can gather useful data about the health of your software, and rapidly put it to use. Catching irregularities before they hit production can be the difference between happy customers singing your praises and those who move on to seek greener, more reliable pastures.

“It’s already a hard job,” Palestine says of building software. “The hard part of building software should be in developing the feature. I work every day to ensure my team has confidence they won’t break anything in production.”