This IS NOT Fine: Putting Out (Code) Fires

So the dumpster is on fire. Again. The site’s down. Your boss’s face is an ever-deepening purple. And you begin debating whether you should join the #incident channel or call an ambulance to deal with his impending stroke.

Fires are never going to stop. We’re human. We miss bugs. Or we fat finger a command — deleting dozens of servers and bringing down S3 in US-EAST-1 for hours — effectively halting the internet. These things happen.

But we can fundamentally change the way we approach fires. And that requires adopting the techniques of industries much older than ours.

Firefighters have clear procedures and a strong hierarchy. The first truck at a scene immediately begins assessing the situation. They evaluate building construction, visible smoke, fire and flow paths. The Incident Commander gives orders to his or her personnel regarding fire attack as well as calling for additional resources, communicating with those not yet at at the scene and preparing an action plan.