Life as a developer can take many spooky turns.

Original art from xkcd


From forum quandaries that haunt you through time, to sneaky bugs you find lurking in your legacy code, the things you encounter as a developer can be downright shocking. Read on for some spooky stories from our own CircleCI developers in honor of the Halloween season:

“I was once working tech support for a VPS provider and told a customer that their disk image was hosed so they should delete it and restore from the backup. The backup was also hosed and we couldn’t try to repair their image because they deleted it…”

“Let everything run as root. Enough said.”

“I once deleted /home on every machine in our fleet while running an Ansible playbook because the rules for the Jinja templates and order of operations for filtering and loops wasn’t clear, so instead of deleting /home/$employee the $employee was the empty string.”

“I worked at a startup once that was 4 people, including myself, and the owner/boss/self-granted-title-of-the-week made his money during the ‘90s before the bubble pop, he was great at marketing but had very little technical know-how. One day there was a bugged release of the app, and when I told him the root cause of the bug later that day his response was ‘Why isn’t there a backup function that works in case of bugs?’ I gave my notice that day.”

“I once worked at a network assurance company that held a release meeting/party where all the executives spent over an hour going on how well the company was doing, then laid off a third of the staff the next day.”

“One of my last companies had a revolving door of architects for about 3 years, and the product went from being pure Clojure/ClojureScript to being Python, Go, Clojure, ClojureScript, Java, JavaScript, and some C++ thrown in for good measure.”

“Let’s not use Kubernetes, and try to roll our own stateless Kubernetes.”

“An old company rendered every microservice configuration file from a master values file and jinja templates, then would deploy the configuration both to zookeeper and directly into the containers running the various services. That worked out at least 1/4 as well as you’d expect it to.”

“I once inherited running a webservice that was lighttp running bash scripts using CGI that parsed env vars like query string and path using sed/awk/cut to extract params, then used touch and rm plus some craftily maintained state on the filesystem instead of using a real DB - they didn’t even use SQLite!”

“Refactoring a monolith into microservices using the monolith as a dependency.”

Have frightful tales of your own? Share them by tweeting your #devhorror stories to @CircleCI