Collin and the Slingbot

In this talk we describe our internal feature-flagging system, which combines Rollout, ZooKeeper, and our in-house campfire chatbot (twke), to transparently enable features for targeted production end-users without disrupting other customers.

Our talk follows the escapades of our intrepid engineer Collin as he cleverly employs feature flagging to manage production traffic flow in order to re-engineer and replace a core component of our production SaaS infrastructure. Like the engineering acrobatics involved in reworking a bridge that must continue to bear traffic, Collin's story is a prolonged, high-stakes, surgical endeavor with a lot of moving parts. It is also a textbook illustration of the multi-disciplinary balance between architecture, programming, and operational ingenuity that exemplifies site reliability engineering in the wild.

Joseph Ruscio is a Co-Founder and the Chief Technology Officer at Librato. He's responsible for the company's technical strategy, product architecture, and hacks on all levels of their vision for the future of monitoring. Joe has 15 years of experience developing distributed systems in startups, academia, and the telecommunications industry and he holds a Masters in Computer Science from Virginia Tech. In his spare time he enjoys snowboarding and obsessing over the details of brewing both coffee and beer. He loves graphs.