(This is part one of a four-part series. Read parts two, three, and four here.)
Software engineers spend most of their waking hours wading through the mud of their predecessors. Only a few are lucky enough to see green fields before conflict transforms the terrain; the rest are shipped to the front (end). There, they languish in trenches as shells of outages explode around them. Progress is usually glacial, though ground can be covered through heroic sprints.
But veterans do emerge, scarred and battle-hardened. They revel in relating their most daring exploits and bugfixes to new recruits. And just as individuals have learned individual lessons about writing code, our industry has learned collective lessons about software development at scale. It’s not always easy to see these larger trends when you’re on the ground — buried in bugs and focusing fire on features. DevOps is one of these larger trends. It’s the unification of two traditionally disparate worlds into one cohesive cycle. But it’s not some recent invention or fad; it’s the result of years of iteration, as engineers have cracked risk into progressively smaller chunks. Not every engineer experienced this process first-hand. And for those who missed the march towards continuous development, there may be gaps in understanding why or how we got here.
George Santayana wrote that “those who cannot remember the past are condemned to repeat it.” He was definitely not thinking about software when he wrote this, but he’s dead now, which means I can quote him out of context. Oh, the joys of Public Domain!
This series will be about the history of software development methodologies — especially where they intersect with traditional best practices. Think of it as The Silmarillion of Silicon Valley, except shorter and with more pictures. Before plunging into this rushing river of time, please note that the presented chronology is both theoretically complete and practically in progress. In other words, even though a term or process might have been coined, it always takes more time for Best Practices to trickle down to Real Products. And trickling down is where we’ll start.
Waterfall
When software development was still young, engineers modeled their work on the hardware world. This is an environment where making mistakes is expensive, so it made sense to iron out kinks before producing a thousand units of a thing. If a toy company, for example, decided that a stuffed moose should also provide witty political commentary, that discovery would need to happen before production started. Otherwise, they’d make thousands of silent moose units, and everybody knows those don’t sell.
In 1970, Dr. Winston Royce wrote an article called Managing the Development of Large Software Systems. This article outlined a process (shown below) that bore an eerie resemblance to the accepted methodology of the hardware world.
The process itself is comprised of several phases, each of which must be completed in its entirety before moving to the next phase — with the exception of minor iteration between adjacent phases. Royce never used the word waterfall anywhere in his paper; in fact, the original genius who attached this word to the process has been lost to history.
But everyone readily accepted it because it made sense: as waterfalls flow downward, so too do the phases of software development. And water does not (usually) travel upward — and then only when coerced. How brilliantly expressive! Never let it be said that software engineers do not have passionate, poetic cores.
The intent behind the waterfall model was to create a methodical, unsurprising production cycle. Its advocates loudly proclaimed its strengths: it generates oodles of documentations before coding even begins; it delivers a Certain Something defined In Advance; it’s always easy to know how much work remains. In theory, these all sound wonderful, but waterfall development suffers when put into practice.
Under Pressure
While Royce believed in the waterfall model conceptually, he thought actual implementation was “risky and invited failure”. This is because of the method’s (in)famous inflexibility. The problem lies in getting each phase exactly right the first time. There is very little room to take lessons from the Testing Phase back to the Design Phase; this would undermine the very goals of waterfall development: predictability, certainty, consistency — all those gluey goodies that keep things slow and steady.
So waterfall development is, ironically, not very fluid. At scale, the model just doesn’t move quickly enough to leverage the speed of software or adapt to the market’s changing needs. And since the schedule reigns supreme, there’s often massive procrastination and subsequent death marches to keep the train on time.
“One does not simply develop software like hardware.” — Dr. Winston Royce
Like your grandparents, waterfall development is motivated by good intents but is kind of out of the technological loop. When projects stick to their waterfall guns, there is often a Grand Piling Up of work. In the software world, this can mean months of design and planning before any coding even happens. By the time it’s finished, the product might not even be relevant or even wanted!
Agile Minds
Software is built on hardware.
And early software took pointers from hardware because it didn’t know any better. So, software developers inherited a development strategy that just wasn’t a great fit for writing code. And as the world accelerated, companies realized they needed to be more nimble.
Waterfall is often depicted as the clown in the court of software development methodologies. We’ve defined it to distance ourselves from the “backward traditionalists” who haven’t managed to move onto modern practices. But the truth is that waterfall development was reality at one point; without it as a baseline, we wouldn’t have been able to iterate ourselves into the future.
Every journey has to start somewhere! In the case of this blog post series, that somewhere was an outdated practice — guaranteed to cause stress and dramatic tension.
But that dramatic tension will be resolved in the next post when we dive into the Agile Movement!
Sources
http://www.base36.com/2012/12/agile-waterfall-methodologies-a-side-by-side-comparison/ https://www.techrepublic.com/article/understanding-the-pros-and-cons-of-the-waterfall-model-of-software-development/ https://en.wikipedia.org/wiki/Waterfall_model
