sponsors
help promote
Get more
Help Promote graphics!
usenix conference policies
Global Analytics in the Face of Bandwidth and Regulatory Constraints
Ashish Vulimiri, University of Illinois at Urbana-Champaign; Carlo Curino, Microsoft; P. Brighten Godfrey, University of Illinois at Urbana-Champaign; Thomas Jungblut, Microsoft; Jitu Padhye and George Varghese, Microsoft Research
Global-scale organizations produce large volumes of data across geographically distributed data centers. Querying and analyzing such data as a whole introduces new research issues at the intersection of networks and databases. Today systems that compute SQL analytics over geographically distributed data operate by pulling all data to a central location. This is problematic at large data scales due to expensive transoceanic links, and may be rendered impossible by emerging regulatory constraints. The new problem of Wide-Area Big Data (WABD) consists in orchestrating query execution across data centers to minimize bandwidth while respecting regulatory constaints. WABD combines classical query planning with novel network-centric mechanisms designed for a wide-area setting such as pseudo-distributed execution, joint query optimization, and deltas on cached subquery results. Our prototype, Geode, builds upon Hive and uses 250 less bandwidth than centralized analytics in a Microsoft production workload and up to 360 less on popular analytics benchmarks including TPC-CH and Berkeley Big Data. Geode supports all SQL operators, including Joins, across global data.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Ashish Vulimiri and Carlo Curino and P. Brighten Godfrey and Thomas Jungblut and Jitu Padhye and George Varghese},
title = {Global Analytics in the Face of Bandwidth and Regulatory Constraints},
booktitle = {12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15)},
year = {2015},
isbn = {978-1-931971-218},
address = {Oakland, CA},
pages = {323--336},
url = {https://www.usenix.org/conference/nsdi15/technical-sessions/presentation/vulimiri},
publisher = {USENIX Association},
month = may
}
connect with us