Service Levels and Error Budgets

100% is almost never the right reliability target for a service, and service level agreements (SLAs) aren't the right tool for SREs to manage a service. These two (apparent) heresies are fundamental to how Google SRE thinks about running large-scale distributed computing services: we set service level objectives (SLOs) expressing how reliable a service needs to be and manage our service to maximize product development and feature velocity within the agreed "error budget."

We'll discuss the differences between indicators, objectives, and agreements; error budgets in practice; and how this brings product managers, product developers, and SREs together in a spirit of peaceful coexistence and cooperation.

Chris Jones is a Site Reliability Engineer for Google App Engine, a cloud platform-as-a-service product serving over 28 billion requests per day. Based in San Francisco, he has previously been responsible for the care and feeding of Google's advertising statistics, data warehousing, and customer support systems. In other lives, Chris has worked in academic IT, analyzed data for political campaigns, and engaged in some light BSD kernel hacking, picking up degrees in Computer Engineering, Economics, and Technology Policy along the way. He's also a licensed professional engineer.

Niall Murphy leads the Ads Site Reliability Engineering team at Google Ireland. He has been involved in the Internet industry for about 20 years, and is currently chairperson of INEX, Ireland's peering hub. He is the author or co-author of a number of technical papers and/or books, including "Site Reliability Engineering" for O' Reilly, and a number of RFCs. He is currently co-writing a history of the Internet in Ireland, and he is the holder of degrees in Computer Science, Mathematics, and Poetry Studies, which is surely some kind of mistake. He lives in Dublin with his wife and two sons.