DevOps didn’t exist when I started as a developer: How this one principle changed my career

When asked to write a “What is DevOps?” post, I decided to take an approach that was different from the many others who have written on the subject. In this post, I’m going to define DevOps based on my decades of experience working in tech. I’ll traverse my professional timeline discussing how DevOps came into my life and expressing how it impacted me and my chosen career.

The 90’s

In the mid 90’s, the internet was in its infancy and the world wasn’t as connected as it is today. People were carrying around little black books which contained handwritten contact information for the people that they wanted to communicate with. They had pagers, aka beepers, used to request callbacks. The dissemination of information was limited to paper mediums such as periodicals and books. People would visit libraries and interact with paper, index card-based cataloging systems to retrieve materials and then physically open the magazine, newspaper, or book to extract the information they desired. This was an absolutely manual process during an analogue age.

I started programming professionally in 1994 and the technology landscape was obviously different than that of today. Most companies were in this weird position where they saw the value of technology, but were often hesitant to invest in it or to adopt it. At that time, servers, cpus, memory, and storage were very expensive. Bandwidth was scarce, limited, and also came at a premium. Most software development and operations (SRE or System Administration) teams developed, tested, and released applications and managed infrastructures manually and, more importantly, in isolation. Data centers had to be designed, constructed, and outfitted with proper power, climate controls, and cabling and inhabited by expensive infrastructure hardware along with the capable, skilled humans to manage it all. From scrappy startup to huge corporate enterprises, they were all rocking some form of a “Data Center”.

My software development experience went a little something like this:

1. Write code in isolation
2. Version new code using the “Save as new…” method
3. Manually compile
4. Manually test the application (usually point and click, submit, validate results are correct)
5. Code reviews were very rare
6. Manually create release packages for deployment
7. Write up release/installation notes for the package
8. Deliver the release package to the ops team via floppy disk, CD, or a shared directory on a network file share
9. Wait…

Waiting on the results of the new release could take anywhere from 4 hours to 10 days. The length of time was dependent on how busy the ops team was and the skill of the operator deploying the new release. There was little communication between the developer and operations teams. In fact, most developer and ops teams were at odds with one another. The animosity usually stemmed from deep frustrations between the teams over the quality of software (untested or weak test coverage) or the lack of urgency in deploying the software to production environments.

As my career progressed, I spent lots of time reflecting on the factors causing the unnecessary animus among the dev and ops teams. I couldn’t understand why these highly talented teams with skilled individuals couldn’t figure out how to work efficiently and cohesively to achieve our shared goals and mission. It was clear that the dev and ops teams were not aligned at all. Both parties had one goal and that was to release software to production. Aside from this goal, there really wasn’t any interest in working together to solve the issues. During this period in technology, the developer and operations relationships were negatively influenced by the disconnect and it fostered toxic cultures. Of course, this wasn’t the case for every team and organization, but I can confidently say that some form of the described culture was present in most teams/orgs.

Agile

I described a sliver of my experience as a software developer early on in my career to give you a general understanding of the attitudes, mindsets, and cultures I was working with. I recognize that I’m not painting the prettiest picture of how things were at that time. Others may have had a completely different experience, but I believe it’s safe to assume elements of the type of dysfunction I experienced existed within a majority of teams.

As time passed and technology evolved, it brought about cheaper, faster hardware and bandwidth that offered increased capabilities. Also during this time, dev teams got smarter about how software was being developed. Teams began to analyze their processes top to bottom and bottlenecks in the software development lifecycle (SDLC) were being exposed and recognized. Teams realized that delivering quality software faster could be achieved by changing their software development processes and improving communication amongst themselves. Many teams and organizations started adopting and implementing Agile Software Development concepts which helped them understand their development deficiencies and enabled them to experiment in incorporating new concepts and ideas that produced new methodologies and strategies. These Agile methodologies included an iterative approach which focused on collaboration, customer feedback, and small, rapid releases.

After successfully operating under Agile principles with various development teams and organizations for a few years, I noticed that teams were producing code much faster than ever before. Developers were collaborating on code in the open and version control systems were critical in teams being able to create, modify, and share code. Over time, dev teams were working smarter and the time to produce quality code dramatically shortened. Code was being written, tested, and packaged at a high velocity. Although the dev team was firing on all cylinders and code was being ferociously slung, we suffered from an unexpected problem. The new code being developed wasn’t being released as fast as it was being produced. We quickly realized that we had only focused on improving our software development processes without proper consideration of the release or deployment process. So we had caches of new code that was not being released to our customers and that, to us developers, was an enormous problem.

My intro to DevOps

I’ve always had a strong connection with the code that I write and I’ve always had an extreme interest in the “Who, How, What, and Where” aspects of my code. I knew that I had to develop genuine, healthy relationships with my ops teams in order to satisfy my borderline obsessive curiosity. Early on in my career, I realized the symbioses between developer and operational teams and naively assumed that other devs and operators shared my assumptions. Admittedly, I was floored when I was exposed to reality. I just couldn’t comprehend why there were “barriers” between these teams. I viewed these “barriers” as excuses for dev and ops teams to control and maintain their siloed empires. I viewed these manufactured barriers between the two camps as a huge suppression of innovation. Since this realization, I’ve always referred to myself as “a developer with operational tendencies” which I feel is a fitting description.

After realizing that all the gains from our Agile endeavour were being utterly hindered by the lack of basic interaction with our operational counterparts, we banded together to form new strategies that would include ops in the SDLC. Now at this time, the term DevOps wasn’t a thing yet, but the spirit of the word was definitely alive and well within dev teams that had adopted and implemented Agile. Many “Agile” teams where in the same boat. The dev teams were on point, rapidly developing software, and they were encountering massive blocking when it came to actually releasing code to production.

Some of my developer colleagues were aware of my good standing with the ops team and suggested that I leverage that relationship to forge a bridge between the two teams. I was a bit concerned about putting my neck out on the line and risking the relationships I had built with the ops team. I didn’t want to damage the relationship over an idea that could be easily misconstrued as a power grab by the dev team. I thought long and hard about how to approach the ops team with this radical new idea. I remember rehearsing the conversation over and over in my head, and when I felt I had it down, I scheduled a meeting with the ops team. The anxiety incrementally grew as the date neared.

The day finally arrived and I opened the conversation with the overly rehearsed pitch I had developed. To my surprise, the ops manager told me to stop talking, then unexpectedly asked me how his team could operate at the same tempo as the dev team. He explained that they had been silently observing our increase in coordination, quality, velocity, and success. He also acknowledged that the ops team collectively realized that their current processes were huge blockers in getting software released faster. I was elated to hear that this was the ops team’s position and that they were willing to work with us to improve both dev and ops processes.

We embarked on our “breaking down barriers” journey which wasn’t simple or easy. To my recollection, devising a lean agile strategy that both teams understood and accepted was the biggest obstacle that we had to overcome. Since the dev team had already been exposed to Agile and had gone through an Agile culture shift, we came across to the ops team as a bit elitist when making suggestions to the ops team. The dev team’s confidence was being perceived as egotistical. Considering the situation and the sensitivity attached to it, we quickly developed communication strategies which helped us to address one another in a manner that minimized the sanctimonious tone that put individuals in defensive postures. Increasing the level of communication between the teams felt like the biggest barrier to topple. Once we figured that out, we were collectively able to function as both individual entities and as a single unit, delivering software in sync.

Over time, the dev and ops teams’ interactions improved, eventually leading to the elimination of the previous disfunction. The teams developed a genuine trust amongst themselves and that’s when the magic began. Developers were educating the operators on their development processes and the minutia related to their stack paradigms. The operators were educating the developers on their release processes and the minutia of their roles in maintaining their infrastructures. These unifying changes between the teams didn’t happen all at once. The changes occurred over time and I attribute the successful culture shifts to unified efforts and the learnings from iterative trials and errors. The teams learned the benefits of communication and transparency. That lead them to better define and understand their individual roles and how they impacted the unit collectively.

What is DevOps?

We had achieved a true DevOps culture amongst our teams before the term “DevOps” was even coined. Years later, after the first time that I had heard the term DevOps, the light bulb went off and I knew exactly what it meant. It was the perfect word to describe what we had achieved.

This is where I get to share my definition of DevOps with you:

DevOps is…

• a concept
• a mindset
• a shared attitude understood and embraced by individuals
• a culture that must be nurtured and iteratively improved
• sharing
• mentoring
• learning
• inclusive and open to all ideas
• iterative
• continuous
• collaborative
• an awesome way to confidently develop and deliver software

DevOps is NOT…

• easily achieved nor implemented
• a product or tool chain
• a job title or role
• a cloud infrastructure provider
• a book
• a technology
• a programming language
• a marketing campaign
• CI/CD
• Kubernetes
• containers
• open source software
• Infrastructure as Code
• automation
• to be trifled with!!!

Integrating more DevOps practices benefits the entire organization, not just the engineering team. It’s important to understand the reasons why some companies don’t adopt DevOps practices efficiently.

Conclusion

As promised, I delivered my personal take on DevOps with a bit of context based on my decades-long experiences. I hope my depiction puts things into a relatable perspective and helps underscore the need for DevOps and the value that it brings to an organization. I had lots of fun recalling my experiences and sharing them with you.

Thanks for reading.