(This is part one of a four-part series. Read parts two, three, and four here.)
Software engineers spend most of their waking hours wading through the mud of their predecessors. Progress is usually glacial, though ground can be covered through heroic sprints.
They emerge, post-deployment, older and hopefully wiser. They revel in relating their most daring exploits and bug fixes to new devs. And just as individuals have learned individual lessons about writing code, our industry has learned collective lessons about software development at scale. It’s not always easy to see these larger trends when you’re on the ground — swatting bugs and focusing on delivering features.
DevOps is one of these larger trends. It’s the unification of two traditionally disparate worlds into one cohesive cycle. But it’s not some recent invention or fad; it’s the result of years of iteration, as engineers have broken down risk into progressively smaller chunks. Not every engineer experienced this process first-hand. And if you missed the earlier march towards continuous development, there may be gaps in your understanding of why and how we got here.
This series will be about the history of software development methodologies — especially where they intersect with traditional best practices.
Waterfall
When software development was still young, engineers modeled their work on the hardware world. This is an environment where making mistakes is expensive, so it made sense to iron out wrinkles before producing a thousand units of a thing. If a toy company decided that a stuffed moose should also promote travel to Canada, that would need to happen before production started. Otherwise, they’d make thousands of silent moose units, and the Canadian travel industry would miss out on a valuable marketing tool.
In 1970, Dr. Winston Royce wrote an article called Managing the Development of Large Software Systems. This article outlined a process with an obvious resemblance to the accepted methodology of the hardware world.
The process itself is comprised of several phases, each of which must be completed in its entirety before moving to the next phase — with the exception of minor iteration between adjacent phases. Royce never used the word waterfall anywhere in his paper; in fact, the name of the person who attached this word to the process has been lost to history.
But everyone readily accepted it because it made sense: as waterfalls flow downward, so too do the phases of software development. And water does not (usually) travel upward — and then only when coerced.
The intent behind the waterfall model was to create a methodical, unsurprising production cycle. Its advocates loudly proclaimed its strengths: it generates pages and pages of documentations before coding even begins; it delivers a product defined in advance; it’s always easy to know how much work remains. In theory, these all sound wonderful, but waterfall development suffers when put into practice.
Under pressure
While Royce believed in the waterfall model conceptually, he thought actual implementation was “risky and invited failure”. This is because of the method’s infamous inflexibility. The problem lies in getting each phase exactly right the first time. There is very little room to take lessons from the testing phase back to the design phase. Doing that would undermine the very goals of waterfall development: predictability, certainty, consistency — everything that keeps things slow and steady.
So waterfall development is, ironically, not very fluid. At scale, the model doesn’t move quickly enough to keep up with the speed of software or adapt to the market’s changing needs. There can be months of design and planning before any coding even happens. Procrastination can result in long hours spent in last-minute panic.
Agile minds
Software is built on hardware.
Early software development duplicated the hardware design process because that process had been so successful. The result was that software developers inherited a development strategy that wasn’t a great fit for writing code. But the truth is that waterfall development was reality at one point; without it as a baseline, we wouldn’t have been able to iterate ourselves into the future. As the world accelerated, companies realized they needed to be more nimble.
Every journey has to start somewhere! In the case of this blog post series, that somewhere was an outdated practice — prone to cause stress and dramatic tension.
But that dramatic tension will be resolved in the next post when we discuss the Agile movement.
