Cluster Management at Google

Cluster management is the term that Google uses to describe how we control the computing infrastructure in our datacenters that supports almost all of our external services. It includes allocating resources to different applications on our fleet of computers, looking after software installations and hardware, monitoring, and many other things. I'll present an overview of some of these systems and introduce Omega, the new cluster-manager tool we are building. Much of the talk will be about exciting challenges that we're facing along the way, driven by the scale at which we operate, an acute awareness of failures, and the drive to provide ever-better service-levels while curbing complexity. We certainly don't have all the answers, but we do have some pretty impressive systems.

John Wilkes has been at Google since 2008, where he is working on cluster management and infrastructure services. He is interested in far too many aspects of distributed systems, but a recurring theme has been technologies that allow systems to manage themselves. In his spare time he continues, stubbornly, trying to learn how to blow glass. http://e-wilkes.com/john