Packet classification is important for applications such as firewalls, intrusion detection, and differentiated services. Existing algorithms for packet classification reported in the literature scale poorly in either time or space as filter databases grow in size. Hardware solutions such as TCAMs do not scale to large classifiers. However, even for large classifiers (say 100,000 rules), any packet is likely to match a few (say 10) rules. Our paper seeks to exploit this observation to produce a scalable packet classification scheme called Aggregated Bit Vector (ABV). Our paper takes the bit vector search algorithm (BV) described in [11] (which takes linear time) and adds two new ideas, recursive aggregation of bit maps and filter rearrangement, to create ABV (which can take logarithmic time for many databases). We show that ABV outperforms BV by an order of magnitude using simulations on both industrial firewall databases and synthetically generated databases.

A classifier consists of a set of rules for classifying packets based on header fields. Because core routers can have fairly large (e.g., 2000 rule) database and must use limited SRAM to meet OC-768 speeds, the best existing classification algorithms (RFC, HiCuts, ABV) are precluded because of the large amount of memory they need. Thus the general belief is that hardware solutions like CAMs are needed, despite the amount of board area and power they consume. In this paper, we provide an alternative to CAMs via an Extended Grid-of-Tries with Path Compression (EGT-PC) algorithm whose worst-case speed scales well with database size while using a minimal amount of memory. Our evaluation is based on real databases used by Tier 1 ISPs, and synthetic databases. EGT-PC is based on a observation that we found holds for all the Tier 1 databases we studied: regardless of database size, any packet matches only a small number of distinct source-destination prefix pairs. The code we wrote for EGT-PC, RFC, HiCuts, and ABV is publicly available [16], providing the first publicly available code to encourage experimentation with classification algorithms.

Abstract—Emerging Internet applications create the need for advanced packet classifiers. We propose a novel multifield classification scheme, called € P g, which exploits the strengths of state-of-the-art memory technologies to provide wire-speed classification performance for OC-192 and beyond, in combination with very high storage efficiency and the support of fast incremental updates. Key features of the new scheme are its ability to adapt to the complexity of a classification rule set, whereas the storage requirements and update dynamics can be tuned at the granularity of individual rules. This makes € P g suitable for a broad spectrum of applications. Index Terms—Associative memories, communication system routing, communication systems, Internet, routing, search methods, table lookup, tree data structures, tree searching.

Routers must perform packet classification at high speeds to efficiently implement functions such as firewalls and diffserv. Classification can be based on an arbitrary number of fields in the packet header. Performing classification quickly on an arbitrary number of fields is known to be difficult, and has poor worst-case complexity.

Many network devices such as routers and firewalls employ caches to take advantage of temporal locality of packet headers in order to speed up packet processing decisions. Traditionally, cache designs trade off time and space with the goal of balancing the overall cost and performance of the device. In this paper, we examine another axis of the design space that has not been previously considered: accuracy. In particular, we quantify the benefits of relaxing the accuracy of the cache on the cost and performance of packet classification caches. Our cache design is based on the popular Bloom filter data structure. This paper provides a model for optimizing Bloom filters for this purpose, as well as extensions to the data structure to support graceful aging, bounded misclassification rates, and multiple binary predicates. Given this, we show that such caches can provide nearly an order of magnitude cost savings at the expense of misclassifying one billionth of packets for IPv6-based caches.

The traffic volume between origin/destination (OD) pairs in a network, known as traffic matrix, is essential for efficient network provisioning and traffic engineering. Existing approaches of estimating the traffic matrix, based on statistical inference and/or packet sampling, usually cannot achieve very high estimation accuracy. In this work, we take a brand new approach in attacking this problem. We propose a novel data streaming algorithm that can process traffic stream at very high speed (e.g., 40 Gbps) and produce traffic digests that are orders of magnitude smaller than the traffic stream. By correlating the digests collected at any OD pair using Bayesian statistics, the volume of traffic flowing between the OD pair can be accurately determined. We also establish principles and techniques for optimally combining this streaming method with sampling, when sampling is necessary due to stringent resource constraints. In addition, we propose another data streaming algorithm that estimates flow matrix, a finer-grained characterization than traffic matrix. Flow matrix is concerned with not only the total traffic between an OD pair (traffic matrix), but also how it splits into flows of various sizes. Through rigorous theoretical analysis and extensive synthetic experiments on real Internet traffic, we demonstrate that these two algorithms can produce very accurate estimation of traffic matrix and flow matrix respectively.

Network processor cache maintains results of previous packet lookup or classification computation for subsequent reuse. Earlier research had shown that unlike standard CPU cache, one can significantly improve the effective coverage of a network processor cache by caching based on ranges of lookup/classification keys rather than individual keys. However, the earlier work focused specifically on reducing the capacity misses and did not address two other important aspects - (a) reducing the conflict miss and (b) the cache consistency issue due to frequent routing table updates. We propose two techniques to minimize the conflict miss. One aims to reduce the deviation in the number of cacheable entries mapped to each cache set, and the other allows different number of cache sets to be associated with different IP address partitions. We present simulation results based upon a large routing table view constructed from several Internet backbone routers and IP packet traces collected from a major edge router in Taiwan. The results shows that the two optimization techniques can individually reduce the cache miss ratio by up to 76% and 45.2%, respectively. We also propose a selective route cache invalidation technique to minimize the performance overhead due to frequent route updates. This technique can reduce the cache miss ratio by up to 79.6% compared to a naive wholecache invalidation scheme when there is a routing table update every ten thousand packet lookups. Our results are promising for network processors used at Internet edge and make a strong case for further research into caching dynamics at the Internet core.

this paper we present the hardware design of a high-speed ATM firewall that does not require the termination of an end-to-end connection in the middle. We propose a novel firewall design philosophy, called Quality of Firewalling (QoF), that applies security measures of different strength to traffic with different risk levels and show how it can be implemented in our firewall. Compared with the traditional firewalls, this ATM firewall performs exactly the same packet-level filtering without compromising the performance and has the same "look and feel" by sitting at the chokepoint between the trusted ATM LAN and untrusted ATM WAN. It is also easy to manage and flexible to use.

Routers must perform packet classi cation at high speeds to e ciently implement functions such as rewalls. The classi-cation can be based on an arbitrary number of pre x and range elds in the packet header. The classi cation required for rewalls is beyond the capabilities o ered by standard Operating System classi ers such as BPF [12], DPF [7], PathFinder [1] and others. In fact, there are theoretical results that show the general rewall classi cation problem has poor worst case cost: for searching over N arbitrary lters using k packet elds, either the worst-case search time is ((log N) k,1) or the worstcase storage is O(N k). In this paper, we re-examine two basic mechanisms that have been dismissed in the literature as being too ine cient: backtracking search and set pruning trees. We nd using real databases that the time for backtracking search ismuch better than the worst case bound; instead of ((logN) k,1), the search time is only roughly twice the optimal search time 1. Similarly, we nd that set pruning trees (using a DAG optimization) have much better storage costs than the worst case bound; it has memory requirements similar to the RFCscheme of Gupta and McKeown [10]. We also propose several new techniques to further improve the two basic mechanisms. Our major ideas are a novel compression algorithm, the ability to trade smoothly between backtracking and set pruning, and algorithms to e ectively make use of hardware if hardware is available. We quantify the performance gain of each technique using real databases. We show that on real rewall databases our schemes, with the accompanying optimizations, are close to optimal in time and storage. 1.

Many Internet functions require classification of each packet based on a number of packet header fields and a set of rules. Packet classification as a general theory problem is inherently hard [5]. While fast network processors have been successfully developed to keep up with wire speeds, the only widening gap between memory access speeds and wire speeds represents an increasingly tough challenge to pure software solutions. Given that, most router vendors favor hardware solutions based on TCAM for its fast and scalable lookup speed. However, as a more complex technology, TCAM is more expensive and more power consuming than RAM-based software solutions. Moreover, TCAM is well known to suffer inefficient range specification [4]. If a rule specifies range clauses on k packet header fields that are m1-bit, m2-bit, · · · , mk-bit wide, respectively, it will take up to Qk i=1 (2mi − 2) TCAM entries to represent the rule. As wire speeds, the size and complexity of rule sets rapidly increase, a TCAM-based solution where all rules are expressed in TCAMs will become increasingly expensive. To be cost efficient, using a fast but small cache is a natural and appealing option. Researchers [1] have proposed to cache recent incoming packets and the corresponding decision to speed up the classification of succeeding packets. However, an observation is that rule sets are much smaller in size and much more static in composition than the flow population observed by routers. In recent studies [2] from a tier-1 ISP, it is reported that a very small number of rules match most of incoming traffic. Therefore, much higher and much more stable hit ratios may be achieved by caching rules instead of caching packets. Notice that stability means robustness against traffic pattern change caused by either malicious attack or other reasons. While the basic idea is conceptually clear, some key prob-