Using Influence to Understand Complex Systems

When a complex production system misbehaves, the debugging technique of first resort is to examine instrumentation logs, which are normally noisy and incomplete. We define a statistical notion of "influence" between components that gracefully handles this situation. Intuitively, two components influence each other if they tend to exhibit surprising behavior around the same time. We show how to efficiently compute and answer queries about influence and how to present results in a useful form as a Structure-of-Influence Graph among components, and we give example applications of these ideas to several production systems, including autonomous vehicles, clusters, and supercomputers.