Rethinking Adaptability in Wide-Area Stream Processing Systems

Adaptability is an important property of stream processing systems since the systems need to maintain low latency and high throughput execution of long-running queries. In a wide-area environment, dynamics are common not only because of the workload variability but also the nature of wide-area network (WAN) bandwidth that frequently changes. In this work, we study the adaptability property of stream processing systems designed for a wide-area environment. Specifically, we (1) discuss the challenges of reconfiguring query executions in a wide-area environment, (2) propose ideas how to adapt existing reconfiguration techniques used in centralized Cloud to a wide-area environment, and (3) discuss the trade-offs between them. A key finding is that the best adaptation technique to use depends on the network conditions, types of query, and optimization metrics.