DMCA
Enabling Real-Time Querying of Live and Historical Stream Data
Cached
Download Links
- [www-vis.lbl.gov]
- [vis.lbl.gov]
- [db.cs.berkeley.edu]
- [db.cs.berkeley.edu]
- DBLP
Other Repositories/Bibliography
| Citations: | 18 - 0 self |
BibTeX
@MISC{Reiss_enablingreal-time,
author = {Frederick Reiss and Kurt Stockinger and Kesheng Wu and Arie Shoshani and Joseph M. Hellerstein},
title = {Enabling Real-Time Querying of Live and Historical Stream Data},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Applications that query data streams in order to identify trends, patterns, or anomalies can often benefit from comparing the live stream data with archived historical stream data. However, searching this historical data in real time has been considered so far to be prohibitively expensive. One of the main bottlenecks is the update costs of the indices over the archived data. In this paper, we address this problem by using our highly-efficient bitmap indexing technology (called FastBit) and demonstrate that the index update operations are sufficiently efficient for this bottleneck to be removed. We describe our prototype system based on the TelegraphCQ streaming query processor and the FastBit bitmap index. We present a detailed performance evaluation of our system using a complex query workload for analyzing real network traffic data. The combined system uses TelegraphCQ to analyze streams of traffic information and FastBit to correlate current behaviors with historical trends. We demonstrate that our system can simultaneously analyze (1) live streams with high data rates and (2) a large repository of historical stream data. 1
Keyphrases
historical stream data real-time querying index update operation main bottleneck historical trend high data rate data stream live stream data live stream complex query workload current behavior update cost archived historical stream data query processor real time historical data prototype system traffic information detailed performance evaluation combined system highly-efficient bitmap large repository fastbit bitmap index real network traffic data
