The Google File System

The Google File System (2003)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Sanjay Ghemawat , Howard Gobioff , Shun-Tak Leung

Citations:

637 - 3 self

BibTeX

@MISC{Ghemawat03thegoogle,
    author = {Sanjay Ghemawat and Howard Gobioff and Shun-Tak Leung},
    title = {The Google File System},
    year = {2003}
}

Bookmark

OpenURL

Abstract

We have designed and implemented the Google File System, a scalable distributed file system for large distributed data-intensive application s. It provides fault tolerance while running on inexpensive commodity hardware, and it delivers high aggregate performance to a large number of clients. While sharing man y of the same goals as previous distributed file systems, our design has been driven by observations of our application workloads and technological environment, both current and anticipated, that reflect a marked departure from some earlier file system assumptions. This has led us to reexamine tradition al choices an d explore radically different design points. The file system has successfully met our storage needs. It is widely deployed within Google as the storage platform for the gen ration and processing of data used by our service as well as research and development efforts that require large data sets. The largest cluster to date provides hundreds of terabytes of storage across thousands of disks on over a thousand machines, and it is concurrently accessed by hundreds of clients. In this paper, we present file system interface extensions designed to support distributed application, discuss many aspects of our design, and report measurements from both micro-benchmarks and real world use.

Citations

760	Scale and performance in a distributed file system - Howard, Kazar, et al. - 1988
675	A case for redundant arrays of inexpensive disks (raid - Patterson, Gibson, et al. - 1988
257	Frangipani: A scalable distributed file system - Thekkath, Mann, et al. - 1997
155	Replication in the Harp file system - Liskov, Ghemawat, et al. - 1991
152	A costeffective, high-bandwidth storage architecture - Gibson, Nagle, et al. - 1998
102	Cluster I/O with River: Making the fast case common - Arpaci-Dusseau, Treuhaft, et al. - 1999
58	The global file system - Soltis, Ruwart, et al. - 1996
4	Swift: using distributed disk stripping to provide high I/O data rates - Cabrera, Long
3	Serverless networkfile systems - Anderson, Dahlin, et al. - 1995
2	GPFS: A shared-diskfile system for large computing clusters - Haskin - 2002