Dynamo: amazon’s highly available key-value store

Dynamo: amazon’s highly available key-value store (2007)

Cached

Download Links

by Giuseppe DeCandia , Deniz Hastorun , Madan Jampani , Gunavardhan Kakulapati , Avinash Lakshman , Alex Pilchin , Swaminathan Sivasubramanian , Peter Vosshall , Werner Vogels

Venue:	IN PROC. SOSP
Citations:	194 - 0 self

BibTeX

@INPROCEEDINGS{DeCandia07dynamo:amazon’s,
    author = {Giuseppe DeCandia and Deniz Hastorun and Madan Jampani and Gunavardhan Kakulapati and Avinash Lakshman and Alex Pilchin and Swaminathan Sivasubramanian and Peter Vosshall and Werner Vogels},
    title = {Dynamo: amazon’s highly available key-value store},
    booktitle = {IN PROC. SOSP},
    year = {2007},
    pages = {205--220},
    publisher = {}
}

Bookmark

OpenURL

Abstract

Reliability at massive scale is one of the biggest challenges we face at Amazon.com, one of the largest e-commerce operations in the world; even the slightest outage has significant financial consequences and impacts customer trust. The Amazon.com platform, which provides services for many web sites worldwide, is implemented on top of an infrastructure of tens of thousands of servers and network components located in many datacenters around the world. At this scale, small and large components fail continuously and the way persistent state is managed in the face of these failures drives the reliability and scalability of the software systems. This paper presents the design and implementation of Dynamo, a highly available key-value storage system that some of Amazon’s core services use to provide an “always-on ” experience. To achieve this level of availability, Dynamo sacrifices consistency under certain failure scenarios. It makes extensive use of object versioning and application-assisted conflict resolution in a manner that provides a novel interface for developers to use.

Citations

3028	H.: Chord: A scalable Peer-To-Peer lookup service for internet applications - Stoica, Morris, et al.
2115	Clocks and the Ordering of Events in a Distributed System - Lamport - 1978
1194	Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems - Rowstron, Druschel
847	Oceanstore: An architecture for global-scale persistent storage - Kubiatowicz, Bindel, et al. - 2000
637	The Google file system - Ghemawat, Gobioff, et al. - 2003
607	Storage Management and Caching in Past, a Large-Scale, Persistent Peer-to-Peer Storage Utility - Rowstron, Druschel - 2001
445	The dangers of replication and a solution - Gray, Helland, et al. - 1996
438	Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the World Wide Web - Karger, Lehman, et al. - 1997
357	SEDA: An architecture for well-conditioned, scalable Internet services - Welsh, Culler, et al. - 2001
352	Managing update conflicts in Bayou, a weakly connected replicated storage system - Terry, Theimer, et al. - 1995
343	Extensible Cluster-Based Scalable Network Services - Fox - 1997
336	FARSITE: Federated, available, and reliable storage for an incompletely trusted environment - Adya, Bolosky, et al. - 2002
289	A majority consensus approach to concurrency control for multiple copy databases - Thomas - 1979
285	Bigtable: A Distributed Storage System for Structured Data - Chang, Dean, et al.
183	A digital signature based on a conventional encryption function - Merkle - 1987
95	An algorithm for concurrency control and recovery in replicated distributed databases - Bernstein, Goodman - 1984
93	Resolving file conflicts in the Ficus file system - Reiher, Heidemann, et al.
92	FAB: Building Distributed Enterprise Disk Arrays from Commodity Components - Saito, Frølund, et al. - 2004
46	Progress-based regulation of low-importance processes - Douceur, Bolosky - 1999
45	Beehive: O(1) lookup performance for power-law query distributions in peer-to-peer overlays - Ramasubramanian, Sirer - 2004
37	On scalable and efficient distributed failure detectors - Gupta, Chandra, et al. - 2001
12	Antiquity: Exploiting a secure log for wide-area distributed storage - WEATHERSPOON, EATON, et al.
9	Coda: A Resilient Distributed File System - Satyanarayanan, Kistler, et al. - 1987