Results 1  10
of
11
An Improved Construction for Counting Bloom Filters
 14th Annual European Symposium on Algorithms, LNCS 4168
, 2006
"... Abstract. A counting Bloom filter (CBF) generalizes a Bloom filter data structure so as to allow membership queries on a set that can be changing dynamically via insertions and deletions. As with a Bloom filter, a CBF obtains space savings by allowing false positives. We provide a simple hashingbas ..."
Abstract

Cited by 41 (3 self)
 Add to MetaCart
Abstract. A counting Bloom filter (CBF) generalizes a Bloom filter data structure so as to allow membership queries on a set that can be changing dynamically via insertions and deletions. As with a Bloom filter, a CBF obtains space savings by allowing false positives. We provide a simple hashingbased alternative based on dleft hashing called a dleft CBF (dlCBF). The dlCBF offers the same functionality as a CBF, but uses less space, generally saving a factor of two or more. We describe the construction of dlCBFs, provide an analysis, and demonstrate their effectiveness experimentally. 1
External perfect hashing for very large key sets
 In Proceedings of the 16th ACM Conference on Information and Knowledge Management (CIKM’07
, 2007
"... A perfect hash function (PHF) h: S → [0, m − 1] for a key set S ⊆ U of size n, where m ≥ n and U is a key universe, is an injective function that maps the keys of S to unique values. A minimal perfect hash function (MPHF) is a PHF with m = n, the smallest possible range. Minimal perfect hash functio ..."
Abstract

Cited by 16 (3 self)
 Add to MetaCart
A perfect hash function (PHF) h: S → [0, m − 1] for a key set S ⊆ U of size n, where m ≥ n and U is a key universe, is an injective function that maps the keys of S to unique values. A minimal perfect hash function (MPHF) is a PHF with m = n, the smallest possible range. Minimal perfect hash functions are widely used for memory efficient storage and fast retrieval of items from static sets. In this paper we present a distributed and parallel version of a simple, highly scalable and nearspace optimal perfect hashing algorithm for very large key sets, recently presented in [4]. The sequential implementation of the algorithm constructs a MPHF for a set of 1.024 billion URLs of average length 64 bytes collected from the Web in approximately 50 minutes using a commodity PC. The parallel implementation proposed here presents the following performance using 14 commodity PCs: (i) it constructs a MPHF for the same set of 1.024 billion URLs in approximately 4 minutes; (ii) it constructs a MPHF for a set of 14.336 billion 16byte random integers in approximately 50 minutes with a performance degradation of 20%; (iii) one version of the parallel algorithm distributes the description of the MPHF among the participating machines and its evaluation is done in a distributed way, faster than the centralized function.
Small subset queries and Bloom filters using ternary associative memories, with applications
 In SIGMETRICS
, 2010
"... Associative memories offer high levels of parallelism in matching a query against stored entries. We design and analyze an architecture which uses a single lookup into a Ternary Content Addressable Memory (TCAM) to solve the subset query problem for small sets, i.e., to check whether a given set (th ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
Associative memories offer high levels of parallelism in matching a query against stored entries. We design and analyze an architecture which uses a single lookup into a Ternary Content Addressable Memory (TCAM) to solve the subset query problem for small sets, i.e., to check whether a given set (the query) contains (or alternately, is contained in) any one of a large collection of sets in a database. We use each TCAM entry as a small Ternary Bloom Filter (each ‘bit ’ of which is one of {0,1,“∗ ”}) to store one of the sets in the collection. Like Bloom filters, our architecture is susceptible to false positives. Since each TCAM entry is quite small, asymptotic analyses of Bloom filters do not directly apply. Surprisingly, we are able to show that the asymptotic false positive probability formula can be safely used if we penalize
Tribica: Trie Bitmap Content Analyzer for HighSpeed Network Intrusion Detection
 Proc. IEEE INFOCOM
, 2007
"... Abstract—Deep packet inspection (DPI) is often used in network intrusion detection and prevention systems (NIDPS), where incoming packet payloads are compared against known attack signatures. Processing every single byte in the incoming packet payload has a very stringent time constraint, e.g., 200 ..."
Abstract

Cited by 8 (4 self)
 Add to MetaCart
Abstract—Deep packet inspection (DPI) is often used in network intrusion detection and prevention systems (NIDPS), where incoming packet payloads are compared against known attack signatures. Processing every single byte in the incoming packet payload has a very stringent time constraint, e.g., 200 ps for a 40Gbps line. Traditional DPI systems either need a large memory space or use special memory such as ternary content addressable memory (TCAM), limiting parallelism, or yielding high cost/power consumption. In this paper, we present a highspeed, singlechip DPI scheme that is scalable and configurable through memory updates. The scheme is based on a novel data structure called TriBiCa (Trie Bitmap Content Analyzer), which provides minimal perfect hashing functionality. It uses a trie structure with a hash function performed at each layer. Branching is determined by the hashing results with an objective to evenly partition attack signatures into multiple groups at each layer. During a query, as an input traverses the trie, an address to a table in the memory that stores all attack signatures is formed and is used to access the signature for an exact match. Due to the small space required, multiple copies of TriBiCa can be implemented on a single chip to perform pipelining and parallelism simultaneously, thus achieving high throughput. We have designed the TriBiCa on a modest FPGA chip, Xilinx Virtex II Pro, achieving 10Gbps throughput without using any external memory. A proofofconcept design is implemented and tested with 1Gbps packet streams. By using today’s stateoftheart FPGAs, a throughput of 40 Gbps is believed to be achievable. Index Terms—TriBiCa, NIDPS, minimal perfect hashing
Practical perfect hashing in nearly optimal space
 Information Systems
"... A hash function is a mapping from a key universe U to a range of integers, i.e., h: U↦→{0, 1,...,m−1}, where m is the range’s size. A perfect hash function for some set S ⊆ U is a hash function that is onetoone on S, where m≥S. A minimal perfect hash function for some set S ⊆ U is a perfect hash ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
A hash function is a mapping from a key universe U to a range of integers, i.e., h: U↦→{0, 1,...,m−1}, where m is the range’s size. A perfect hash function for some set S ⊆ U is a hash function that is onetoone on S, where m≥S. A minimal perfect hash function for some set S ⊆ U is a perfect hash function with a range of minimum size, i.e., m=S. This paper presents a construction for (minimal) perfect hash functions that combines theoretical analysis, practical performance, expected linear construction time and nearly optimal space consumption for the data structure. For n keys and m=n the space consumption ranges from 2.62n to 3.3n bits, and for m=1.23n it ranges from 1.95n to 2.7n bits. This is within a small constant factor from the theoretical lower bounds of 1.44n bits for m=n and 0.89n bits for m=1.23n. We combine several theoretical results into a practical solution that has turned perfect hashing into a very compact data structure to solve the membership problem when the key set S is static and known in advance. By taking into account the memory hierarchy we can construct (minimal) perfect hash functions for over a billion keys in 46 minutes using a commodity PC. An open source implementation of the algorithms is available
Blooming trees for minimal perfect hashing
 in Proceedings of the Global Communications Conference (GLOBECOM). IEEE, Nov 2008
"... Abstract—Hash tables are used in many networking applications, such as lookup and packet classification. But the issue of collisions resolution makes their use slow and not suitable for fast operations. Therefore, perfect hash functions have been introduced to make the hashing mechanism more effici ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Abstract—Hash tables are used in many networking applications, such as lookup and packet classification. But the issue of collisions resolution makes their use slow and not suitable for fast operations. Therefore, perfect hash functions have been introduced to make the hashing mechanism more efficient. In particular, a minimal perfect hash function is a function that maps a set of n keys into a set of n integer numbers without collisions. In literature, there are many schemes to construct a minimal perfect hash function, either based on mathematical properties of polynomials or on graph theory. This paper proposes a new scheme which shows remarkable results in terms of space consumption and processing speed. It is based on an alternative to Bloom Filters and requires about 4 bits per key and 12.8 seconds to construct a MPHF with 3.8 × 109 elements. I.
A 10Gbps HighSpeed SingleChip Network Intrusion Detection and Prevention System
"... (NIDPSs) are vital in the fight against network intrusions. NIDPSs search for certain malicious content in network traffic (i.e., signatures). Comparing all traffic to these signatures is a challenge for highspeed networks. In this paper, we present the implementation of a 10Gbps hardware NIDPS an ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(NIDPSs) are vital in the fight against network intrusions. NIDPSs search for certain malicious content in network traffic (i.e., signatures). Comparing all traffic to these signatures is a challenge for highspeed networks. In this paper, we present the implementation of a 10Gbps hardware NIDPS and related design issues. This goal of signature detection at highspeed is achieved using a single FPGA, without any external memory. We also implemented and tested a proofofconcept system with 1Gbps traffic. A database to store and a web server to display the intrusion alerts from the NIDPS were also developed for this system. I.
Packet Classification Algorithms
"... This paper deals with packet classification in computer networks. Classification is the key task in many networking devices, most notably packet filters – firewalls. This paper therefore concerns the area of computer security. The paper is focused on highspeed networks with the bandwidth of 100 Gb/ ..."
Abstract
 Add to MetaCart
This paper deals with packet classification in computer networks. Classification is the key task in many networking devices, most notably packet filters – firewalls. This paper therefore concerns the area of computer security. The paper is focused on highspeed networks with the bandwidth of 100 Gb/s and beyond. Generalpurpose processors cannot be used in such cases, because their performance is not sufficient. Therefore, specialized hardware is used, mainly ASICs and FPGAs. Many packet classification algorithms designed for hardware implementation were presented, yet these approaches are not ready for very highspeed networks. This paper addresses the design of new highspeed packet classification algorithms, targeted for the implementation in dedicated hardware.
Network Intrusion Detection and Prevention Systems
"... (NIDPSs) are critical for network security. The Deep Packet Inspection (DPI) operation consumes a significant amount of resources in NIDPS. This is because to detect malicious activity DPI searches a database of signatures for each byte of every packet. In this paper, we develop a highly spaceeffic ..."
Abstract
 Add to MetaCart
(NIDPSs) are critical for network security. The Deep Packet Inspection (DPI) operation consumes a significant amount of resources in NIDPS. This is because to detect malicious activity DPI searches a database of signatures for each byte of every packet. In this paper, we develop a highly spaceefficient data structure for hardware realization of Minimal Perfect Hash Functions (MPHFs). This data structure is simple to construct, requires 7n bits to represent the MPHF for a set of n keys and allows highspeed DPI.
HighSpeed Network Intrusion Detection and Prevention
, 2007
"... Today more than ever, gaining unauthorized access to network resources is appetizing
for malicious intruders. Network Intrusion Detection and Prevention Systems
(NIDPSs) are vital against these intrusions. For NIDPS, Deep Packet Inspection (DPI)
 comparing incoming packet payloads against known att ..."
Abstract
 Add to MetaCart
Today more than ever, gaining unauthorized access to network resources is appetizing
for malicious intruders. Network Intrusion Detection and Prevention Systems
(NIDPSs) are vital against these intrusions. For NIDPS, Deep Packet Inspection (DPI)
 comparing incoming packet payloads against known attack signatures  is the most timeconsuming
operation which is even more challenging for highspeed networks where a
byte of data arrives at every 200 ps from each 40Gbps link. Traditional DPI systems use
either offchip randomaccess or contentaddressable memory, neither of which is up to
the speed challenge. To increase throughput by reducing offchip access, onchip filters
such as Bloom Filters (BF) are also used. However, in this thesis, we show that BFs have
shortcomings when implemented on hardware. Hence, we propose Aggregated Bloom Filters
(ABF) to increase the throughput and scalability of hardware BFs. ABF leverages the
query mechanism for hardware BFs by removing redundant hash calculations and redundant
onchip memory accesses for higher throughput. ABF also improves scalability by
aggregating small distributed BFs to a single BF for better onchip memory utilization.
ABF shows sevenfold improvement in the average query throughput and four times less
memory usage compared to previous hardware BFs for NIDPS.
In the second part of this thesis, we expand our focus to optimize the memory
usage for the entire DPI and eliminate the offchip memory, completely. We propose a new
data structure called TriBiCa (Trie Bitmap Content Analyzer), a minimal perfect hashing
scheme for hardware, to achieve this goal. We have designed the TriBiCa for Xilinx Virtex
II Pro FPGA chip, achieving 10Gbps NIDPS throughput. A proofofconcept design is
implemented and tested with 1Gbps packet streams. By using today’s stateoftheart
FPGAs, we believe a throughput of 40 Gbps is achievable.
NIDPS can easily be evaded by fragmentation of attack packets. The straightforward
defragmentation method is not applicable at highspeeds due to high memory requirement.
In the final part of this thesis, this multipacket signature detection problem
is addressed using a defragmentationfree, spaceefficient solution. A new data structure,
Prefix Bloom Filters (PBFs), is proposed to significantly reduce the storage requirement of
the problem.