Benchmarking in The Dark: On the Absence of Comprehensive Edge Datasets

Application workloads are used for system analysis, design, optimization, and evaluation. As systems and their applications evolve, new workloads are required to reflect their characteristics and bottlenecks. Edge-computing systems are a new model with unique characteristics, including heterogeneous and geodistributed components and complex failure patterns. Their workload characteristics are also unique, including unpredictable load and user mobility, combined with stringent latency and bandwidth requirements. Supporting these workloads would require resource management and allocation policies that take into account a rich set of attributes.

In this paper, we show that currently available datasets include only partial subsets of these attributes. Moreover, having been collected on existing systems, they do not reflect the unique characteristics of edge systems and applications. At the same time, current edge systems in early deployment stages are not ready for the collection of representative application workloads. We show how to bridge the gap between required and available datasets by workload composition: we combine attributes from several available datasets to create realistic representations of edge systems, their users, and their workloads.