EngineeringLast Updated Nov 21, 20236 min read

A Brief History of DevOps, Part III: Automated Testing and Continuous Integration

(This is part three of a four-part series. Read parts one, two, and four here.)

Welcome back. In the last tutorial in this series, we discussed the many movements that led to Agile methodology. Towards the end, we foreshadowed how constant validation would play a role in this chapter. This time, we’ll be discussing two processes near and dear to our hearts: automated testing and continuous integration.

Automated testing

We’ve already seen how the Agile movement was inspired by moving away from heavy documentation and towards lighter chunks of work. There was a collective revelation: risk is unavoidable, so you might as well minimize that risk by cutting it into small chunks.

As people grew more comfortable with failure, they created places where failure was not only acceptable, it was encouraged. The rise of testing frameworks like JUnit and Cucumber embodied this new attitude toward risk. Tools don’t precede processes; we create them to optimize existing processes. This meant that engineers were spending enough time writing tests to warrant the creation of these frameworks.

Extreme Programming was the primary influence behind this new enthusiasm. Kent Beck and Ron Jeffries had championed test-driven development with a convincing argument: “If bugs are going to happen (and they will), engineers should fix at least some of them while they still remember the code.” Because code soon evaporates from an engineer’s mind as they move on to new projects.

Writing tests before the code forced engineers to think about how they wanted software to work. In many ways, this practice served the same purpose as the exhaustive documentation of the waterfall years: tests specified how things should be. But automated tests went one step further by enforcing these requirements on each code change.

Engineers discovered that, by running tests more frequently, they increased the odds of catching bugs and preventing catastrophes. That meant checking as you go instead of at the end. This might seem obvious today but, at the time, it was a fundamentally different way of developing software.

And it’s from this focus on testing that continuous integration was born.

On the origin of continuous integration

The DevOps timeline isn’t as straightforward as we’d like it to be.

Rather than a discrete progression from the Agile Era to the Age of Continuous Integration, the two concepts are actually contemporaries. The phrase continuous integration (CI for short) is actually older than many of Agile’s ancestors. It was coined in 1994 by Grady Booch.

Booch is notable for developing the Unified Modeling Language (UML) with two colleagues: Ivar Jacobson and James Rumbaugh. UML is a great example of the overzealous design phase from which the Agile Alliance tried to distance itself, but otherwise has no place in this story.

The phrase itself appears in Booch’s book, Object-Oriented Analysis and Design with Applications:

“The needs of the micro process dictate that many more internal releases to the development team will be accomplished, with only a few executable releases turned over to external parties. These internal releases represent a sort of continuous integration of the system, and exist to force closure of the micro process.”

While the emphasis is my doing, the rest is directly from Booch’s head. He also writes that testing should “be a continuous activity during the development process”, so from early on, testing and continuous integration were closely intertwined.

Pay close attention to Booch’s choice of words: he describes the “needs of the micro process”, and this is a recognition of the pace of software development. He’s not discussing normal-sized or macro processes. Instead, these became micro processes because the slices of delivery were being cut ever more thinly.

Spot the difference

The Agile movement itself could be described as a grand reduction of feedback loops in software. Continuous integration is a natural (though more process-oriented) extension of this reduction.

When done well, continuous integration acts as a kind of unending course correction — a formalized application of the belief that uncertainty is the one certainty. While it has much in common with its Agile cousins, continuous integration differs in scale. The Agile movement provides a bundle of tactics that makes an organization more responsive; continuous integration is a strategic shift that fundamentally changes an organization’s culture.

But these distinctions quickly become blurry. When Kent Beck and Ron Jeffries created Extreme Programming in 1996, they officially adopted continuous integration as one of its twelve core practices. They even took it further, advocating multiple integrations per day.

ThoughtWorks and CruiseControl

In 2000, one of the founding members of the Agile Alliance became an internal advocate for continuous integration at ThoughtWorks. His name is Martin Fowler, and he’s rather famous. Fowler points out that continuous integration isn’t conceptually tied to specific software; developers can practice CI without a tool dedicated to it. But personalization comes at the cost of time, money, and the ability to build a network of users.

Practicing CI can be made easier with a server built specifically for daily builds and continuous testing. And in 2001, another ThoughtWorker named Matthew Foemmel built this and dubbed it CruiseControl. The tool is open-source and still around, though there are fewer maintainers these days.

CruiseControl was important. It helped bring a best practice into reality, and its public nature encouraged adoption across the industry. It didn’t have a ton of responsibility: watch for changes in an application’s codebase and, if necessary, create a new build and run it through a suite of tests to verify that everything actually still works.

Hudson and Jenkins

While CruiseControl was a relatively simple tool, it represented a commitment to both software and chronological integrity. Soon, other contenders were sprouting up around the concept like daffodils. Hudson, another CI tool written in Java, was released in 2005 but didn’t really start to overtake CruiseControl until 2008.

That changed. Hudson’s author, Kohsuke Kawaguchi, had been working at Sun Microsystems, which Oracle purchased in 2010. Like CruiseControl, Hudson was free and open-source, so when Oracle announced its plans to trademark and commercialize it, the Hudson community fled the scene in 2011 under the new name of Jenkins, which continues strong to this day.

(Best) practice makes perfect

There are many places to read about the fallout of that split, so we won’t cover them here. Instead, let’s take a step back and discuss how these tools (and disagreements) represent the industry’s rapid acceptance of a best practice.

In just a decade, developers formulated, proposed, implemented, and commercialized a suggested process. The fact that Oracle was willing to put up a fight indicates that the idea and its implementation have value. And that value has percolated across the software world until it’s become so well-ingrained that the question has shifted from “Are you continuously integrating?” to “When and how often are you continuously integrating?”

For a while, Facebook’s motto was “Move Fast and Break Things”. As the software industry has matured, engineers have realized that you can’t actually move that quickly if everything is breaking. That’s why Facebook changed its motto to “Move Fast With Stable Infra”.

Conclusion

Continuous integration is the first step on the path to balancing speed and stability. Catching problems before they get out of hand demonstrates a shift in how we think about risk. It’s the logical next step on the agile escalator, and the reason it’s an escalator in this metaphor is because the journey never ends.

In theory, at least. In practice, this series is going to end in the next part, when we discuss continuous delivery and deployment.