4th USENIX OSDI Symposium
[Technical Program]
| Pp. 319–332 of the Proceedings |  |
Next: 1. Introduction
Scalable, Distributed Data Structures
for Internet Service Construction
Steven D. Gribble, Eric A. Brewer, Joseph M. Hellerstein, and David Culler
The University of California at Berkeley
{gribble,brewer,jmh,culler}@cs.berkeley.edu
Abstract
This paper presents a new persistent data management layer designed to
simplify cluster-based Internet service construction. This self-managing
layer, called a distributed data structure (DDS), presents a conventional
single-site data structure interface to service authors, but partitions and
replicates the data across a cluster. We have designed and implemented a
distributed hash table DDS that has properties necessary for Internet
services (incremental scaling of throughput and data capacity, fault
tolerance and high availability, high concurrency, consistency, and
durability). The hash table uses two-phase commits to present a coherent
view of its data across all cluster nodes, allowing any node to service any
task. We show that the distributed hash table simplifies Internet service
construction by decoupling service-specific logic from the complexities of
persistent, consistent state management, and by allowing services to
inherit the necessary service properties from the DDS rather than having to
implement the properties themselves. We have scaled the hash table to a
128 node cluster, 1 terabyte of storage, and an in-core read throughput of
61,432 operations/s and write throughput of 13,582 operations/s.
Next: 1. Introduction
gribble@cs.berkeley.edu
|
