DMCA
Natjam: Design and evaluation of eviction policies for supporting priorities and deadlines in mapreduce clusters (2013)
| Venue: | in SoCC. ACM |
| Citations: | 5 - 1 self |
BibTeX
@INPROCEEDINGS{Cho13natjam:design,
author = {Brian Cho and Muntasir Rahman and Tej Chajed and Indranil Gupta and Cristina Abad and Nathan Roberts and Philbert Lin},
title = {Natjam: Design and evaluation of eviction policies for supporting priorities and deadlines in mapreduce clusters},
booktitle = {in SoCC. ACM},
year = {2013}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract This paper presents Natjam, a system that supports arbitrary job priorities, hard real-time scheduling, and efficient preemption for Mapreduce clusters that are resource-constrained. Our contributions include: i) exploration and evaluation of smart eviction policies for jobs and for tasks, based on resource usage, task runtime, and job deadlines; and ii) a work-conserving task preemption mechanism for Mapreduce. We incorporated Natjam into the Hadoop YARN scheduler framework (in Hadoop 0.23). We present experiments from deployments on a test cluster, Emulab and a Yahoo! Inc. commercial cluster, using both synthetic workloads as well as Hadoop cluster traces from Yahoo!. Our results reveal that Natjam incurs overheads as low as 7%, and is preferable to existing approaches.
