CIEL: a universal execution engine for distributed data-flow computing
| Venue: | in Proceedings of the 8th USENIX Symposium on Networked System Design and Implementation (NSDI). USENIX |
| Citations: | 19 - 6 self |
BibTeX
@INPROCEEDINGS{Murray_ciel:a,
author = {Derek G. Murray and Malte Schwarzkopf and Christopher Smowton and Steven Smith and Anil Madhavapeddy and Steven Hand},
title = {CIEL: a universal execution engine for distributed data-flow computing},
booktitle = {in Proceedings of the 8th USENIX Symposium on Networked System Design and Implementation (NSDI). USENIX},
year = {}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
This paper introduces CIEL, a universal execution engine for distributed data-flow programs. Like previous execution engines, CIEL masks the complexity of distributed programming. Unlike those systems, a CIEL job can make data-dependent control-flow decisions, which enables it to compute iterative and recursive algorithms. We have also developed Skywriting, a Turingcomplete scripting language that runs directly on CIEL. The execution engine provides transparent fault tolerance and distribution to Skywriting scripts and highperformance code written in other programming languages. We have deployed CIEL on a cloud computing platform, and demonstrate that it achieves scalable performance for both iterative and non-iterative algorithms. 1
