MapReduce: Simplified data processing on large clusters.

DMCA

MapReduce: Simplified data processing on large clusters. (2004)

Cached

Download Links

by Jeffrey Dean , Sanjay Ghemawat

Venue:	In Proceedings of the Sixth Symposium on Operating System Design and Implementation (OSDI-04),
Citations:	3437 - 3 self

BibTeX

@INPROCEEDINGS{Dean04mapreduce:simplified,
    author = {Jeffrey Dean and Sanjay Ghemawat},
    title = {MapReduce: Simplified data processing on large clusters.},
    booktitle = {In Proceedings of the Sixth Symposium on Operating System Design and Implementation (OSDI-04),},
    year = {2004}
}

OpenURL

Abstract

Abstract MapReduce is a programming model and an associated implementation for processing and generating large data sets. Programs written in this functional style are automatically parallelized and executed on a large cluster of commodity machines. The run-time system takes care of the details of partitioning the input data, scheduling the program's execution across a set of machines, handling machine failures, and managing the required inter-machine communication. This allows programmers without any experience with parallel and distributed systems to easily utilize the resources of a large distributed system. Our implementation of MapReduce runs on a large cluster of commodity machines and is highly scalable: a typical MapReduce computation processes many terabytes of data on thousands of machines. Programmers find the system easy to use: hundreds of MapReduce programs have been implemented and upwards of one thousand MapReduce jobs are executed on Google's clusters every day.

DMCA