The Importance of Modularity and Reasoning in Scalable Robust Systems

The last decade or two have seen a wild growth of large-scale systems that have strong responsiveness requirements. Such systems include cloud services as well as sensor networks. Their scalability and reliability requirements mandate that these systems are both sharded and replicated. Also, these systems evolve quickly as a result of changes in workload, adding functionality, deploying new hardware, and so on. While these systems are useful, they can behave in erratic ways and it is not clear that one can build mission- and life-critical systems this way.

This talk is inspired by work that I’m doing within the ARPA-E GENI program to modernize the power grid using cloud infrastructure. I will survey some of the techniques available to build scalable systems in a modular and reasoned fashion. I will discuss some of the experience I have working with formal methods to derive provably correct building blocks and provably correct transformations that improve trust in the responsiveness or reliability of distributed systems. Finally, I will talk about some of the open questions that remain.

Robbert van Renesse is a Principal Research Scientist in the Department of Computer Science at Cornell University. He received a Ph.D. from the Vrije Universiteit in Amsterdam in 1989. After working at AT&T Bell Labs in Murray Hill he joined Cornell in 1991. He was associate editor of IEEE Transactions on Parallel and Distributed Systems from 1997 to 1999, and he is currently associate editor for ACM Computing Surveys. His research interests include the fault tolerance and scalability of distributed systems. Van Renesse is an ACM Fellow.