Embrace Variables So Your Customers Don't Have To

Release Engineering can be a science. Like any science, there is a goal of controlling for variables to understand how change impacts the results of the overall system (in this case the software delivery pipeline). At Puppet Labs, our goal is to control for variances on operating systems, package managers, service managers, et al, so that our customers don’t have to. But to achieve this, we must purposely add variables into our release train. We currently build and test for more than 60 operating system targets, and that number is only increasing. When you ship an abstraction layer, you have have to test and build for all of the underlying components, and at our scale that must be automatic and developer serviceable.

I'll give you an overview of our build system world, and discuss the challenges, rewards, and horrors of trying to provide fast feedback to hungry development teams across dozens of operating systems and network devices. I’ll cover using continuous delivery processes and ideals to ship an on-premise product, what metrics we’ve found the most useful for decision making, and, of course, share a few failures you can laugh at and not repeat in your own environments.

Michael Stahnke is Director of Engineering Services at Puppet Labs. Previously, he was the Community Manager and the Release Manager, where he built out the Release Engineering team. He came to Puppet Labs from Caterpillar, Inc. where he was an Infrastructure Architect, system administration team lead, and open source evangelist. Michael also helped get the Extra Packages for Enterprise Linux (EPEL) repository off the ground in 2006, and is the author of Pro OpenSSH (Apress, 2005).