## Efficient IP table lookup via adaptive stratified trees with selective reconstructions (2004)

Venue: | 12TH EUROPEAN SYMP ON ALGORITHMS |

Citations: | 1 - 0 self |

### BibTeX

@MISC{Pellegrini04efficientip,

author = {Marco Pellegrini and Giordano Fusco},

title = {Efficient IP table lookup via adaptive stratified trees with selective reconstructions },

year = {2004}

}

### OpenURL

### Abstract

IP address lookup is a critical operation for high bandwidth routers in packet switching networks such as Internet. The lookup is a non-trivial operation since it requires searching for the longest prefix, among those stored in a (large) given table, matching the IP address. Ever increasing routing tables size, traffic volume and links speed demand new and more efficient algorithms. Moreover, the imminent move to IPv6 128-bit addresses will soon require a rethinking of previous technical choices. This article describes a the new data structure for solving the IP table look up problem christened the Adaptive Stratified Tree (AST). The proposed solution is based on casting the problem in geometric terms and on repeated application of efficient local geometric optimization routines. Experiments with this approach have shown that in terms of storage, query time and update time the AST is at a par with state of the art algorithms based on data compression or string manipulations (and often it is better on some of the measured quantities).

### Citations

323 | Skip Lists: A Probabilistic Alternative to Balanced Trees
- Pugh
- 1990
(Show Context)
Citation Context ...egree is uniform and is decided beforehand; no optimization, local or global is done. Ergun et al. [Sharp et al. 2001] and [Ergun et al. 2001] use the fast reconfiguration capabilities of skip lists [=-=Pugh 1990-=-] to adapt on-line the search data structure to the modifications of traffic patterns. A number of recent papers have addressed the IP lookup problem with the aim of improving the worst case asymptoti... |

285 | Computational Geometry: An Introduction Through Randomized Algorithms - Mulmuley - 1994 |

201 |
Design and implementation of an efficient priority queue, Math. Systems Theory 10
- Boas, Kaas, et al.
- 1977
(Show Context)
Citation Context ...d fast qtries) using storage O(n) achieving query time O(min{log log U, √ log U}). Stratified trees of van Emde Boas achieve query time O(log log U) using storage O(U) in the original paper [van Emde =-=Boas et al. 1977-=-]. Storage was reduced afterwards to O(nU ǫ ) in [Johnson 1982], and to O(n) in [van Emde Boas 1977] when n > |U|/ log c |U|. Willard in [Willard 2000] lowers the lower limit for ataining linear stora... |

176 |
PATRICIA - Practical Algorithm To Retrieve Information Coded
- Morrison
- 1968
(Show Context)
Citation Context ...ournal of the ACM, Vol. V, No. N, Month 20YY.s4 · M. Pellegrini and G. Fusco The classical data structure to solve the longest matching prefix problem is the binary trie [Knuth 1973]. Patricia Tries [=-=Morrison 1968-=-] take advantage of common sub-sequences to compress certain paths in a binary trie, thus saving both in search time and storage. The first IP Lookup algorithm in [Sklower 1991] is based on the Patric... |

172 | Small forwarding tables for fast routing lookups
- Degermark, Brodnik, et al.
- 1997
(Show Context)
Citation Context ...in [Buchsbaum et al. 2003] is different: the target is a good average time for random traffic with no limit on storage, thus up to 3 Mbytes are used to store a table of 118K entries. Dagemark et al. [=-=Degermark et al. 1997-=-] use data compression techniques to store compactly parts of the prefix-tree representing the set of prefixes. At present this technique achieves in practice the lowest use of storage. Crescenzi et a... |

164 |
The Art of Computer
- Knuth
- 1998
(Show Context)
Citation Context ...ay generate some confusion. Journal of the ACM, Vol. V, No. N, Month 20YY.s4 · M. Pellegrini and G. Fusco The classical data structure to solve the longest matching prefix problem is the binary trie [=-=Knuth 1973-=-]. Patricia Tries [Morrison 1968] take advantage of common sub-sequences to compress certain paths in a binary trie, thus saving both in search time and storage. The first IP Lookup algorithm in [Sklo... |

149 | Scalable high speed IP routing lookups
- Waldvogel, Varghese, et al.
- 1997
(Show Context)
Citation Context ...of level compression by increasing the out-degree of each node of the trie as long as at least a large fraction (called the filling factor) of the sub-trees of a node are non-empty. Waldvogel et al. [=-=Waldvogel et al. 1997-=-] organize the prefixes in groups of prefixes of same length and each group is stored in a hash table for fast membership testing. The search is essentially a binary search by prefix length over the h... |

120 | Fast address lookups using controlled prefix expansion
- Srinivasan, Varghese
- 1999
(Show Context)
Citation Context ...search is essentially a binary search by prefix length over the hash tables (a quite similar technique is in [Willard 1983]). The number of hash tables can be reduced by padding some of the prefixes [=-=Srinivasan and Varghese 1999-=-]. Good hashing strategies are studied in [Broder and Mitzenmacher 2001a]. In [Srinivasan and Varghese 1999] a method called variable stride trie is described. Here a trie is built but the number of b... |

107 | Trade-offs for Packet Classification
- Feldmann, Muthukrishnan
(Show Context)
Citation Context ... is called the geometric (or locus) view as opposed to the previous string view. The geometric approach has been proposed in [Warkhede et al. 2004] and [Lampson et al. 1999]; it has been extended in [=-=Feldmann and Muthukrishnan 2000-=-] and in several recent papers [Thorup 2003; Kaplan et al. 2003]. It has been used in [Buchsbaum et al. 2003] to prove formally the equivalence among several different problems. Important asymptotic w... |

102 | G.: Ip-address lookup using lc-tries
- Nilsson, Karlsson
- 1999
(Show Context)
Citation Context ...nces to compress certain paths in a binary trie, thus saving both in search time and storage. The first IP Lookup algorithm in [Sklower 1991] is based on the Patricia Trie idea. Nilsson and Karlsson [=-=Nilsson and Karlsson 1999-=-] add the concept of level compression by increasing the out-degree of each node of the trie as long as at least a large fraction (called the filling factor) of the sub-trees of a node are non-empty. ... |

88 | Preserving order in a forest in less than logarithmic time - Boas, P - 1990 |

84 |
Log-logarithmic worst-case range queries are possible in space θ(n), Information Processing Letters 17 (2
- Willard
- 1983
(Show Context)
Citation Context ... of same length and each group is stored in a hash table for fast membership testing. The search is essentially a binary search by prefix length over the hash tables (a quite similar technique is in [=-=Willard 1983-=-]). The number of hash tables can be reduced by padding some of the prefixes [Srinivasan and Varghese 1999]. Good hashing strategies are studied in [Broder and Mitzenmacher 2001a]. In [Srinivasan and ... |

83 | IP lookups using multiway and multicolumn search
- Lampson, Srinivasan, et al.
- 1998
(Show Context)
Citation Context ...egment on a line containing a query point. This is called the geometric (or locus) view as opposed to the previous string view. The geometric approach has been proposed in [Warkhede et al. 2004] and [=-=Lampson et al. 1999-=-]; it has been extended in [Feldmann and Muthukrishnan 2000] and in several recent papers [Thorup 2003; Kaplan et al. 2003]. It has been used in [Buchsbaum et al. 2003] to prove formally the equivalen... |

74 | Linear approximation of shortest superstrings
- BLUM, JIANG, et al.
- 1991
(Show Context)
Citation Context ...ze the memory occupation are found via dynamic programming. In [Buchsbaum et al. 2003] the trie is augmented by compressing all leaves and internal nodes via shortest common super-string compression [=-=Blum et al. 1994-=-]. It should be noticed that while in [Srinivasan and Varghese 1999] the model states a 1 Mbyte cap on storage and tries to attain the best worst case time within the cap (which is also our model), th... |

68 | Using multiple hash functions to improve ip lookups - Mitzenmacher, Broder, et al. - 2000 |

67 | Longest prefix matching using bloom filters
- Dharmapurikar, Krishnamurthy, et al.
- 2003
(Show Context)
Citation Context ...e is a quite large portion of the recent research on the IP lookup problem that uses specialized hw architectures different form the hw model adopted in this paper (e.g. [Hasan and Vijaykumar 2005], [=-=Dharmapurikar et al. 2003-=-], [Taylor et al. 2003], [Liu and Lea 2001], [Wuu and Pin 2001]). For this reason we do not comment further on these results. 9. ON THE STATIC AND DYNAMIC PREDECESSOR PROBLEM On the predecessor proble... |

61 | Optimal bounds for the predecessor problem
- Beame, Fich
- 1999
(Show Context)
Citation Context ...son 1982], and to O(n) in [van Emde Boas 1977] when n > |U|/ log c |U|. Willard in [Willard 2000] lowers the lower limit for ataining linear storage to n > |U|/(log U) c·loglog |U| ). Beame and Fich [=-=Beame and Fich 1999-=-] have shown that the predecessor problem among n input data from an integer universe U can be found using O(n) storage and query time � � � �� O min (log log U)(log log n) , log log log U log n log l... |

60 | A tree-based packet routing table for berkeley UNIX
- Sklower
- 1991
(Show Context)
Citation Context ...1973]. Patricia Tries [Morrison 1968] take advantage of common sub-sequences to compress certain paths in a binary trie, thus saving both in search time and storage. The first IP Lookup algorithm in [=-=Sklower 1991-=-] is based on the Patricia Trie idea. Nilsson and Karlsson [Nilsson and Karlsson 1999] add the concept of level compression by increasing the out-degree of each node of the trie as long as at least a ... |

59 | Tree bitmap: hardware/software IP lookups with incremental updates
- Eatherton, Varghese, et al.
(Show Context)
Citation Context ...e that this technique is “generic” and as such could be applied directly also to AST’s. Data Structures stressing storage requirements. Elaborating on ideas in [Degermark et al. 1997] Earthon at al. [=-=Eatherton et al. 2004-=-] propose the Tree bitmap data structure that can be adapted to several memory models. For the memory model adopted in this paper (L2 and L1 cache levels) they are able to re-map the tables in [Degerm... |

44 |
A priority queue in which initialization and queue operations take O(log log D) time
- Johnson
- 1982
(Show Context)
Citation Context ...og U, √ log U}). Stratified trees of van Emde Boas achieve query time O(log log U) using storage O(U) in the original paper [van Emde Boas et al. 1977]. Storage was reduced afterwards to O(nU ǫ ) in [=-=Johnson 1982-=-], and to O(n) in [van Emde Boas 1977] when n > |U|/ log c |U|. Willard in [Willard 2000] lowers the lower limit for ataining linear storage to n > |U|/(log U) c·loglog |U| ). Beame and Fich [Beame an... |

38 | M.: Time-space trade-offs for predecessor search - Patrascu, Thorup - 2006 |

37 | Faster deterministic sorting and searching in linear space
- Andersson
- 1996
(Show Context)
Citation Context ...(n) and performing queries in time O(min{log log U, 1 + log n/ loglog U}). A randomized method of Anderson [Andersson 1995] uses storage O(n) and has expected query time O( √ log n). Subsequently in [=-=Andersson 1996-=-], still using linear storage, the method has been made deterministic and the query time improved to � � �� �log log n O min n, log log U log log n, log log n + . log log U The problem of the optimali... |

35 | Sublogarithmic searching without multiplications
- Andersson
- 1995
(Show Context)
Citation Context ...eame and Fich but worth recalling. Raman [Raman 1995] gives a data structure using storage O(n) and performing queries in time O(min{log log U, 1 + log n/ loglog U}). A randomized method of Anderson [=-=Andersson 1995-=-] uses storage O(n) and has expected query time O( √ log n). Subsequently in [Andersson 1996], still using linear storage, the method has been made deterministic and the query time improved to � � �� ... |

29 | 2000), “Near Optimal Routing Lookups with Bounded Worst Case Performance
- Gupta, Prabhakar, et al.
- 2000
(Show Context)
Citation Context ...e contrasted with several techniques in literature where a global optimality criterion is sought usually via much more expensive dynamic programming techniques, such as in [Cheung and McCanne 1999], [=-=Gupta et al. 2000-=-], [Srinivasan and Varghese 1999], and [Buchsbaum et al. 2003]. Ioannidis, Grama and Atallah [Ioannidis et al. 2005] recently proposed a global reconstruction method for tries that relies on a reducti... |

29 | Examining computational geometry, van Emde Boas trees, and hashing from the perspective of the fusion tree - Willard - 2000 |

26 |
New trie data structures which support very fast search operations
- Willard
- 1984
(Show Context)
Citation Context ...eral dynamic, or can made so with some extra machinery, however, since we are interested in the query time vs. storage performance, we will not dwell on such dynamic features. Willard [Willard 1983] [=-=Willard 1984-=-] gives data structures (fast y-tries and fast qtries) using storage O(n) achieving query time O(min{log log U, √ log U}). Stratified trees of van Emde Boas achieve query time O(log log U) using stora... |

25 | Scalable IP Lookup for Internet Routers
- Taylor, Turner, et al.
- 2003
(Show Context)
Citation Context ... the recent research on the IP lookup problem that uses specialized hw architectures different form the hw model adopted in this paper (e.g. [Hasan and Vijaykumar 2005], [Dharmapurikar et al. 2003], [=-=Taylor et al. 2003-=-], [Liu and Lea 2001], [Wuu and Pin 2001]). For this reason we do not comment further on these results. 9. ON THE STATIC AND DYNAMIC PREDECESSOR PROBLEM On the predecessor problem. Consider the follow... |

15 | Dynamic pipelining: making IP-lookup truly scalable
- Hasan, Vijaykumar
- 2005
(Show Context)
Citation Context ...pancy. Hw based methods. There is a quite large portion of the recent research on the IP lookup problem that uses specialized hw architectures different form the hw model adopted in this paper (e.g. [=-=Hasan and Vijaykumar 2005-=-], [Dharmapurikar et al. 2003], [Taylor et al. 2003], [Liu and Lea 2001], [Wuu and Pin 2001]). For this reason we do not comment further on these results. 9. ON THE STATIC AND DYNAMIC PREDECESSOR PROB... |

15 | A dynamic lookup scheme for bursty access patterns
- Ergun, Mittra, et al.
- 2001
(Show Context)
Citation Context ...s akin to a fixed stride k-ways trie with a very large branching factor at the root. The branching degree is uniform and is decided beforehand; no optimization, local or global is done. Ergun et al. [=-=Sharp et al. 2001-=-] and [Ergun et al. 2001] use the fast reconfiguration capabilities of skip lists [Pugh 1990] to adapt on-line the search data structure to the modifications of traffic patterns. A number of recent pa... |

14 | IP address lookup made fast and simple
- Crescenzi, Dardini, et al.
- 1999
(Show Context)
Citation Context ...e data compression techniques to store compactly parts of the prefix-tree representing the set of prefixes. At present this technique achieves in practice the lowest use of storage. Crescenzi et al. [=-=Crescenzi et al. 1999-=-] instead start from a full table representation of the lookup function then apply a data compression technique that reduces the storage to acceptable levels in practice while requiring only 3 memory ... |

14 | Space efficient dynamic stabbing with fast queries - Thorup - 2003 |

13 | Efficient Construction of Multibit Tries for IP Lookup
- Sahni, Kim
- 2001
(Show Context)
Citation Context ...Varghese 1999]. Thus the other quality measure that need to be measured is the memory consumption. For k = 3 and variable stride trie, for a table of 35K prefixes, the reported storage is 677 Kbytes [=-=Sahni and Kim 2003-=-, Table 4, page 676], which is reasonably close to the storage bound reported in [Srinivasan and Varghese 1999]. With the same data set, a value k = 2 would lower the search time but push up the stora... |

12 |
Dynamic rectangular intersection with priorities
- Kaplan, Molad, et al.
- 2003
(Show Context)
Citation Context ...iew. The geometric approach has been proposed in [Warkhede et al. 2004] and [Lampson et al. 1999]; it has been extended in [Feldmann and Muthukrishnan 2000] and in several recent papers [Thorup 2003; =-=Kaplan et al. 2003-=-]. It has been used in [Buchsbaum et al. 2003] to prove formally the equivalence among several different problems. Important asymptotic worst case bounds have been found following the geometric point ... |

12 |
Searching unindexed and nonuniformly generated files in log log N time
- Willard
- 1985
(Show Context)
Citation Context ... function, the height of the N-tree is bounded by a small constant. This result is interesting since it introduces distribution dependent guarantees beyond the standard uniform distribution. Willard [=-=Willard 1985-=-] gives an O(log log n) expected time algorithm to search on data drawn from a smooth distribution. Comparison of AST and predecessor search data structures. The AST doe not have provable worst case b... |

10 | A B-Tree Dynamic Router-Table Design
- Lu, Sahni
(Show Context)
Citation Context ...last criterion in update time. Thus the AST and the result in [Sahni and Kim 2004] are not comparable because they imply a different ranking of relevant performance measures. Similarly, Lu and Sahni [=-=Lu and Sahni 2005-=-] propose a variant of the B-tree to handle efficiently updates. Experimental data reported cannot be compared directly with ours since the raw timing reported in [Lu and Sahni 2005] comprises accesse... |

9 | Fast prefix matching of bounded strings
- Buchsbaum, Fowler, et al.
- 2003
(Show Context)
Citation Context ...sed in [Warkhede et al. 2004] and [Lampson et al. 1999]; it has been extended in [Feldmann and Muthukrishnan 2000] and in several recent papers [Thorup 2003; Kaplan et al. 2003]. It has been used in [=-=Buchsbaum et al. 2003-=-] to prove formally the equivalence among several different problems. Important asymptotic worst case bounds have been found following the geometric point of view, but, currently, the algorithms for I... |

9 | Efficient dynamic lookup for bursty access patterns - Sahni, Kim - 2003 |

8 |
Searching and sorting real numbers
- Ehrlich
- 1981
(Show Context)
Citation Context ...ile the storage is O(nk/s lnk + n). Journal of the ACM, Vol. V, No. N, Month 20YY.s24 · M. Pellegrini and G. Fusco Previous results with distribution dependent guarantees. N-trees defined by Ehrlich [=-=Ehrlich 1981-=-] capture in a formal way the probably older idea of recursive bucketing where the number of buckets depends on the number of data items in each bucket. Results in [Ehrlich 1981] show that the expecte... |

8 | Efficient construction of fixed-stride multibit tries for IP lookup
- Sahni, Kim
- 2001
(Show Context)
Citation Context ... a slower query time than the AST (or other specialized methods such as [Srinivasan and Varghese 1999]). Improved construction of fixed and variable stride tries. Sahni and Kim in a series of papers [=-=Sahni and Kim 2001-=-; 2002; 2003] propose new formulations of the dynamic programming underlying the construction of the fixed stride and variable stride tries in [Srinivasan and Varghese 1999]. The emphasis is on improv... |

8 | Efficient construction of variable-stride multibit tries for IP lookup - Sahni, Kim - 2002 |

7 |
Analysis of N-trees
- Tamminen
- 1983
(Show Context)
Citation Context ...s on the number of data items in each bucket. Results in [Ehrlich 1981] show that the expected construction time is O(n) when the data are drawn uniformly at random from the universe. Later Tamminen [=-=Tamminen 1983-=-] showed that for any data set drawn from a distribution with a bounded and Lebesgue integrable probability density function, the height of the N-tree is bounded by a small constant. This result is in... |

6 | O(log n) dynamic router-tables for prefixes and ranges - Lu, Sahni - 2004 |

5 | Packet classification via improved space decomposition techniques - Geraci, Pellegrini, et al. - 2005 |

5 | Adaptive data structures for ip lookups
- Ioannidis, Grama, et al.
- 2003
(Show Context)
Citation Context ...ch more expensive dynamic programming techniques, such as in [Cheung and McCanne 1999], [Gupta et al. 2000], [Srinivasan and Varghese 1999], and [Buchsbaum et al. 2003]. Ioannidis, Grama and Atallah [=-=Ioannidis et al. 2005-=-] recently proposed a global reconstruction method for tries that relies on a reduction to the knapsack problem and therefore suggests the use of knapsack approximation methods. In [Ioannidis et al. 2... |

3 | Prefix- and interval-partitioned dynamic IP router-tables
- Lu, Kim, et al.
- 2005
(Show Context)
Citation Context ...similar storage bounds but cannot handle updates directly. The Fat-AST would supports dynamic operations but uses roughly 50% more memory on such input data. Partitioning schemes. Lu, Kiam and Sahni [=-=Lu et al. 2005-=-] propose several schemes for an initial partition of the data set into disjoint subsets. The rationale is to be able to apply for each subset a more traditional data structure (e.g. variable stride t... |

3 | Dynamic IP router-tables using highest-priority matching - Lu, Sahni |

3 | Improved data structures for predecessor queries in integer sets
- Raman
- 1995
(Show Context)
Citation Context ...g U ) operations, while with cells of O(log U) C bits each, Ω( � log n/ loglog n) operations are needed. There are several results not as strong as those of Beame and Fich but worth recalling. Raman [=-=Raman 1995-=-] gives a data structure using storage O(n) and performing queries in time O(min{log log U, 1 + log n/ loglog U}). A randomized method of Anderson [Andersson 1995] uses storage O(n) and has expected q... |

2 | Internet packet fileter management and rectangle geometry - Eppstein, Muthukrishnan - 2001 |

2 |
Fast IP Table Lookup and Memory Reduction
- Liu, Lea
(Show Context)
Citation Context ...n the IP lookup problem that uses specialized hw architectures different form the hw model adopted in this paper (e.g. [Hasan and Vijaykumar 2005], [Dharmapurikar et al. 2003], [Taylor et al. 2003], [=-=Liu and Lea 2001-=-], [Wuu and Pin 2001]). For this reason we do not comment further on these results. 9. ON THE STATIC AND DYNAMIC PREDECESSOR PROBLEM On the predecessor problem. Consider the following rather general f... |

2 |
Fast Update Algorithm for IP Forwarding Table Using
- Sun, Sahni, et al.
- 2004
(Show Context)
Citation Context ...P Table Lookup via Adaptive Stratified Trees · 21 memory consumption and fast updates at the cost of having to visit paths of length O(log n) in order to solve IP-lookup queries. Sun, Sahni and Zhao [=-=Sun et al. 2004-=-] propose a general technique for exploiting independent sets in the input data on a multi-processor parallel architecture to speed up update operations. A very recent data structure Sundström and Lar... |