DMCA
Learn more, sample less: Control of volume and variance in network measurement (2005)
Cached
Download Links
- [www.research.att.com]
- [www.research.att.com]
- [www2.research.att.com]
- [nickduffield.net]
- [www2.research.att.com]
- DBLP
Other Repositories/Bibliography
| Venue: | IEEE Transactions in Information Theory |
| Citations: | 72 - 14 self |
BibTeX
@ARTICLE{Duffield05learnmore,,
author = {Nick Duffield and Carsten Lund and Mikkel Thorup},
title = {Learn more, sample less: Control of volume and variance in network measurement},
journal = {IEEE Transactions in Information Theory},
year = {2005},
volume = {51},
pages = {1756--1775}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract — This paper deals with sampling objects from a large stream. Each object possesses a size, and the aim is to be able to estimate the total size of an arbitrary subset of objects whose composition is not known at the time of sampling. This problem is motivated from network measurements in which the objects are flow records exported by routers and the sizes are the number of packet or bytes reported in the record. Subsets of interest could be flows from a certain customer or flows from a worm attack. This paper introduces threshold sampling as a sampling scheme that optimally controls the expected volume of samples and the variance of estimators over any classification of flows. This paper provides algorithms for dynamic control of sample volumes and evaluate them on flow data gathered from a commercial IP network. The algorithms are simple to implement and robust to variation in network conditions. The work reported here has been applied in the measurement infrastructure of the commercial IP network. To not have employed sampling would have entailed an order of magnitude greater capital expenditure to accommodate the measurement traffic and its processing.
