## Counter braids: A novel counter architecture for per-flow measurement (2008)

### Cached

### Download Links

- [www.stanford.edu]
- [www.stanford.edu]
- [www.stanford.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | In ACM SIGMETRICS’08 |

Citations: | 40 - 5 self |

### BibTeX

@INPROCEEDINGS{Lu08counterbraids:,

author = {Yi Lu and Andrea Montanari and Sarang Dharmapurikar and Abdul Kabbani and Balaji Prabhakar},

title = {Counter braids: A novel counter architecture for per-flow measurement},

booktitle = {In ACM SIGMETRICS’08},

year = {2008}

}

### OpenURL

### Abstract

Fine-grained network measurement requires routers and switches to update large arrays of counters at very high link speed (e.g. 40 Gbps). A naive algorithm needs an infeasible amount of SRAM to store both the counters and a flow-tocounter association rule, so that arriving packets can update corresponding counters at link speed. This has made accurate per-flow measurement complex and expensive, and motivated approximate methods that detect and measure only the large flows. This paper revisits the problem of accurate per-flow measurement. We present a counter architecture, called Counter Braids, inspired by sparse random graph codes. In a nutshell, Counter Braids “compresses while counting”. It solves the central problems (counter space and flow-to-counter association) of per-flow measurement by“braiding”a hierarchy of counters with random graphs. Braiding results in drastic space reduction by sharing counters among flows; and using random graphs generated on-the-fly with hash functions avoids the storage of flow-to-counter association. The Counter Braids architecture is optimal (albeit with a complex decoder) as it achieves the maximum compression rate asymptotically. For implementation, we present a lowcomplexity message passing decoding algorithm, which can recover flow sizes with essentially zero error. Evaluation on Internet traces demonstrates that almost all flow sizes are recovered exactly with only a few bits of counter space per flow.

### Citations

8593 |
Elements of Information Theory
- Cover, Thomas
- 1991
(Show Context)
Citation Context ...des[4]. From the information theoretic perspective, the design of an efficient counting scheme and a good flow size estimation is equivalent to the design of an efficient compressor, or a source code =-=[8]-=-. However, the network measurement problem imposes a stringent constraint on such a code: each time the size of a flow changes (because a new packet arrives), a small number of operations must be suff... |

1745 | Compressed sensing
- Donoho
- 2006
(Show Context)
Citation Context ...comparison of the two figures clearly shows a great reduction in space. 1.3 Related Theoretical Literature Compressed Sensing. The idea of Counter Braids is thematically related to compressed sensing =-=[6, 11]-=-, whose central innovation is summarized by the following quote: Since we can “throw away” most of our data and still be able to reconstruct the original with no perceptual loss (as we do with ubiquit... |

1470 | Space/Time Trade-offs in Hash Coding with Allowable Errors
- Bloom
- 1970
(Show Context)
Citation Context ...roximately one in 40 packets causes a flow to be set up [10]), these memory accesses do not create a bottleneck. Flows that span boundaries of measurement epochs can be identified using a Bloom Filter=-=[3]-=-. Finally, we evaluated the algorithm by measuring flow sizes in packets. The algorithm can be used to measure flow sizes in bytes. Since most byte-counting is really the counting of byte-chunks (e.g.... |

1175 | Factor graphs and the sum-product algorithm
- Kschischang, Frey, et al.
- 2001
(Show Context)
Citation Context ...totic compression rate. The full proof is contained in [17] and we state the theorem in the appendix for completeness. The large alphabet size also makes iterative message passing decoding algorithms =-=[15]-=-, such as Belief Propagation, highly complex to implement, as BP passes probabilities rather than numbers. In this paper, we present a novel message passing decoding algorithm of low complexity that i... |

1127 | Self-similarity in World Wide Web traffic: evidence and possible causes - Crovella, Bestavros - 1997 |

893 | Low-density parity-check codes
- Gallager
- 1963
(Show Context)
Citation Context ...yered non-linear structure and a message passing reconstruction algorithm. Sparse random graph codes. Counter Braids is methodologically inspired by the theory of low-density parity check (LDPC) codes=-=[13, 21]-=-. See also related literatures on Tornado codes[18] and Fountain codes[4]. From the information theoretic perspective, the design of an efficient counting scheme and a good flow size estimation is equ... |

840 | Near optimal signal recovery from random projections: Universal encoding strategies
- Candès, Tao
- 2006
(Show Context)
Citation Context ...comparison of the two figures clearly shows a great reduction in space. 1.3 Related Theoretical Literature Compressed Sensing. The idea of Counter Braids is thematically related to compressed sensing =-=[6, 11]-=-, whose central innovation is summarized by the following quote: Since we can “throw away” most of our data and still be able to reconstruct the original with no perceptual loss (as we do with ubiquit... |

385 | A digital fountain approach to reliable distribution of bulk data
- Byers, Luby, et al.
- 1998
(Show Context)
Citation Context ...rse random graph codes. Counter Braids is methodologically inspired by the theory of low-density parity check (LDPC) codes[13, 21]. See also related literatures on Tornado codes[18] and Fountain codes=-=[4]-=-. From the information theoretic perspective, the design of an efficient counting scheme and a good flow size estimation is equivalent to the design of an efficient compressor, or a source code [8]. H... |

316 | New directions in traffic measurement and accounting
- Estan, Varghese
- 2002
(Show Context)
Citation Context .... For example, Cisco’s Netflow [1] counts both 5-tuples and per-prefix flows based on sampling, which introduces a significant 9% relative error even for large flows and more errors for smaller flows =-=[12]-=-. Juniper Networks introduced filter-based accounting [2] to count a limited set of flows predefined manually by operators. The“sample-and-hold”solution proposed by Estan and Varghese in [12], while a... |

297 | An improved data stream summary: The count-min sketch and its applications
- CORMODE, S
(Show Context)
Citation Context ...the degree distribution of each layer. The optimized space is close to the information theoretic limit, enabling CB to fit into small SRAM. Count-Min Sketch. Like Counter Braids, the Count-Min sketch =-=[7]-=- for data stream applications is also a random hashbased structure. With Count-Min, each flow hashes to and updates d counters; the minimum value of the d counters is retrieved as the flow estimate. T... |

228 | Practical Loss-Resilient Codes
- Luby, Mitzenmacher, et al.
- 1997
(Show Context)
Citation Context ...truction algorithm. Sparse random graph codes. Counter Braids is methodologically inspired by the theory of low-density parity check (LDPC) codes[13, 21]. See also related literatures on Tornado codes=-=[18]-=- and Fountain codes[4]. From the information theoretic perspective, the design of an efficient counting scheme and a good flow size estimation is equivalent to the design of an efficient compressor, o... |

175 |
Modern Coding Theory
- Richardson, Urbanke
- 2008
(Show Context)
Citation Context ...yered non-linear structure and a message passing reconstruction algorithm. Sparse random graph codes. Counter Braids is methodologically inspired by the theory of low-density parity check (LDPC) codes=-=[13, 21]-=-. See also related literatures on Tornado codes[18] and Fountain codes[4]. From the information theoretic perspective, the design of an efficient counting scheme and a good flow size estimation is equ... |

70 | Data streaming algorithms for efficient and accurate estimation of flow distribution
- Kumar, Sung, et al.
- 2004
(Show Context)
Citation Context ...ize. Thus, Count-Min is better at handling online queries than CB. Structurally related to Counter Braids (random hashing of flows into counters and a recovery algorithm) is the work of Kumar et. al. =-=[16]-=-. The goal of that work is to estimate the flow size distribution and not the actual flow sizes, which is our aim. In Section 2, we define the goals of this paper and outline our solution methodology.... |

40 | Efficient implementation of a statistics counter architecture
- Ramabhadran, Varghese
(Show Context)
Citation Context ...2.5 Gbps) trace data show that are about 900, 000 distinct flow 5-tuples in a 5-minute interval. On 40-Gbps links, there can easily be an excess of a million dis-per flow (a standard vendor practice =-=[20]-=-), 8 MB of SRAM is needed for counter space alone. 2. Flow-to-counter association rule. The set of active flows varies over time, and the flow-to-counter association rule needs to be dynamically const... |

27 | Robust TCP stream reassembly in the presence of adversaries
- Dharmapurikar, Paxson
- 2005
(Show Context)
Citation Context ...when a flow is established; for example, when a “SYN” packet arrives. Since flows are established much less frequently than packet arrivals (approximately one in 40 packets causes a flow to be set up =-=[10]-=-), these memory accesses do not create a bottleneck. Flows that span boundaries of measurement epochs can be identified using a Bloom Filter[3]. Finally, we evaluated the algorithm by measuring flow s... |

27 |
Design of a novel statistics counter architecture with optimal space and time efficiency
- ZHAO, XU, et al.
- 2006
(Show Context)
Citation Context ...or updating the DRAM counters so that no SRAM counter overflows in between two DRAM updates. The algorithm analyzed in [22] was subsequently improved by Ramabhadran and Varghese [20] and Zhao et. al. =-=[23]-=-. This reduced the algorithm complexity, making it feasible to use a small SRAM with 5 bits per flow to count flow sizes in packets (not bytes). However, all the papers above suffer from the following... |

24 |
Noiseless data compression with low density parity check codes
- Caire, Shamai, et al.
(Show Context)
Citation Context ...ource codes (such as the Lempel-Ziv algorithm), where changing a single bit in the source stream may completely alter the compressed version. We find that the class of source codes dual to LDPC codes =-=[5]-=- work well under this constraint; using features of these codes makes CB a good “incremental compressor.” There is a problem in using the design of LDPC codes for network measurement: with the heavy-t... |

13 |
Passive traffic measurement for IP operations,” in The Internet as a Large-Scale Complex System
- Grossglauser, Rexford
- 2005
(Show Context)
Citation Context ... Keywords Statistics Counters, Network Measurement, Message Passing Algorithms 1. INTRODUCTION There is an increasing need for fine-grained network measurement to aid the management of large networks =-=[14]-=-. Network measurement consists of counting the size of a logical entity called “flow”, at an interface such as a router. A flow is a sequence of packets that satisfy a common set of rules. For instanc... |

12 | Analysis of a Statistics Counter Architecture
- Shah, Iyer, et al.
- 2001
(Show Context)
Citation Context ...ting using a hybrid SRAM-DRAM architecture, and (ii) approximate counting by exploiting the heavy-tail nature of flow size distribution. We review these approaches below. Exact counting. Shah et. al. =-=[22]-=- proposed and analyzed a hybrid architecture, taking the first step towards an implementable large-scale counter array. The architecture consists of shallow counters in fast SRAM and deep counters in ... |

6 | Detailed Network Measurements Using Sparse Graph Counters: The Theory
- Lu, Montanari, et al.
- 2007
(Show Context)
Citation Context ...f hash functions. 2. Incremental compression of flow sizes as packets arrive; only a small number (e.g. 3) of counters are accessed at each packet arrival. 3. Asymptotic optimality. We have proved in =-=[17]-=- that Counter Braids (CB), with an optimal (but NP-hard) decoder, has an asymptotic compression rate matching the information theoretic limit. The result is surprising since CB forms a restrictive fam... |

2 |
Constraint satisfaction networks
- Mézard, Montanari
(Show Context)
Citation Context ...ct 1. Consider a bipartite graph with n flow nodes and m = βn counters nodes, where each flow node connects to k uniformly sampled counter nodes. It is a forest with high probability iff β ≥ k(k − 1) =-=[19]-=-. Assume the bipartite graph is a forest. Since the flow nodes have degree k > 1, the leaves of the trees have to be counter nodes. Theorem 1. For any flow node i belonging to a tree component in the ... |

1 |
networks solutions for network accounting
- Juniper
(Show Context)
Citation Context ...d per-prefix flows based on sampling, which introduces a significant 9% relative error even for large flows and more errors for smaller flows [12]. Juniper Networks introduced filter-based accounting =-=[2]-=- to count a limited set of flows predefined manually by operators. The“sample-and-hold”solution proposed by Estan and Varghese in [12], while achieving high accuracy, measures only flows that occupy m... |