DMCA
Stork data scheduler: mitigating the data bottleneck in e-science (1949)
| Venue: | In Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences |
| Citations: | 3 - 2 self |
BibTeX
@INPROCEEDINGS{Kosar49storkdata,
author = {Tevfik Kosar and Mehmet Balman and Esma Yildirim and Sivakumar Kulasekaran and On Ross},
title = {Stork data scheduler: mitigating the data bottleneck in e-science},
booktitle = {In Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences},
year = {1949},
pages = {3254--3267}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
In this paper, we present the Stork Data Scheduler as a solution for mitigating the data bottleneck in e-science and data-intensive scientific discovery. Stork focuses on planning, scheduling, monitoring and management of data placement tasks and application-level end-to-end optimization of networked I/O for petascale distributed e-Science applications. Unlike existing approaches, Stork treats data resources and the tasks related to data access and movement as first class entities just like compu-tational resources and compute tasks, and not simply the side effect of computation. Stork provides unique features such as aggregation of data transfer jobs considering their source and destination addresses, and an application-level throughput estima-tion and optimization service. We describe how these two features are implemented in Stork and their effects on end-to-end data transfer performance.
Keyphrases
stork data scheduler data bottleneck compute task destination address application-level end-to-end optimization data resource optimization service end-to-end data transfer performance first class entity data placement task unique feature side effect data access application-level throughput estima-tion data-intensive scientific discovery data transfer job compu-tational resource e-science application
