Workshop Program

When systems fail in the field, logged data are frequently the only evidence available for support engineers and developers to assess and diagnose the underlying cause. Consequently, the efficacy of such logging data is a matter of significant practical importance. We have empirically studied tens of thousands of log messages and hundreds of production failures from several widely-used systems, and built several tools for log automation and postmortem log analysis. In this talk, I will summarize our experiences on exploring questions such as "How much do log messages really help in debugging?", "Are they good enough?", "What are the opportunities for improving log qualities?", "Can we automatically improve log messages?", and "How can we automate the log inference?" I will also discuss where the greatest opportunities for impact are likely to be found in the future.

This talk is based on joint work with: Y. Zhou, P. Huang, S. Park, J. Zheng, H. Mai, Y. Liu, M. Lee, X. Tang, W. Xiong, L. Tan, S. Savage, and S. Pasupathy.

Ding Yuan is a graduating Ph.D. candidate in the University of Illinois at Urbana-Champaign and a visiting student at the University of California, San Diego. He will join the University of Toronto as an assistant professor in 2013. His research focuses on practical approaches for failure diagnosis via log messages. He has received two ASPLOS best paper nominations, an ACM SIGSOFT Distinguished Paper Award, an Outstanding Teaching Assistant Award, and a Saburo Muroga Fellowship. His research systems on failure diagnosis has been requested for release by large vendors including Cisco, EMC, Huawei, NetApp, and Qualcomm.

System data is abundant, yet data-driven decision making is currently more of an art than a science. Many organizations rely on data analysis for problem detection and diagnosis, but the process continues to be custom and ad hoc. In this paper, we examine the analytics process undertaken by users to mine large data sets, and try to characterize these searches by the operations performed. Furthermore, we take a first stab at a methodical process to automatically suggest operations based on statistical analysis of previous searches performed.

In this paper we describe Vayu, a system for managing cloud applications from the performance, availability and capacity standpoints. The system automatically learns the behavior of cloud applications and the remediation actions required to avoid and resolve problems that may arise in the application by detecting and creating signatures of problems and mapping them to a finite set of automated remediation actions available in cloud environments. Vayu is based on a set of algorithms; anomaly detection, problem signature and similarity and classification based action learning. 

Problem diagnosis and debugging in distributed environments such as the cloud and popular distributed systems frameworks has been a hard problem. We explore an evaluation of a novel way of debugging distributed systems, such as the MapReduce framework, by using system calls. Performance problems in such systems can be hard to diagnose and to localize to a specific node or a set of nodes. Additionally, most debugging systems often rely on forms of instrumentation and signatures that sometimes cannot truthfully represent the state of the system (logs or application traces for example). We focus on evaluating the performance debugging of these frameworks using a low level of abstraction - system calls. By focusing on a small set of system calls, we try to extrapolate meaningful information on the control flow and state of the framework, providing accurate and meaningful automated debugging.

A promising way to capture the characteristics of changing traffic is to extract significant flow clusters in traffic. However, clustering flows by 5-tuple requires flow matching in huge flow attribute spaces, and thus, is difficult to perform on the fly. We propose an efficient yet flexible flow aggregation technique for monitoring the dynamics of network traffic. Our scheme employs two-stage flow-aggregation. The primary aggregation stage is for efficiently processing a huge volume of raw traffic records. It first aggregates each attribute of 5-tuple separately, and then, produces multi-dimensional flows by matching each attribute of a flow to the resulted aggregated attributes. The secondary aggregation stage is for providing flexible views to operators. It performs multi-dimensional aggregation with the R-tree algorithm to produce concise summaries for operators. We report our prototype implementation and preliminary results using traffic traces from backbone networks.

Tracing mechanisms in distributed systems give important insight into system properties and are usually sampled to control overhead. At Google, Dapper [8] is the always-on system for distributed tracing and performance analysis, and it samples fractions of all RPC traffic. Due to difficult implementation, excessive data volume, or a lack of perfect foresight, there are times when system quantities of interest have not been measured directly, and Dapper samples can be aggregated to estimate those quantities in the short or long term. Here we find unbiased variance estimates of linear statistics over RPCs, taking into account all layers of sampling that occur in Dapper, and allowing us to quantify the sampling uncertainty in the aggregate estimates. We apply this methodology to the problem of assigning jobs and data to Google datacenters, using estimates of the resulting cross-datacenter traffic as an optimization criterion, and also to the detection of change points in access patterns to certain data partitions.