## Novel Architectures for P2P Applications: the Continuous-Discrete Approach (2007)

### Cached

### Download Links

- [www.wisdom.weizmann.ac.il]
- [www.wisdom.weizmann.ac.il]
- [research.microsoft.com]
- [www.wisdom.weizmann.ac.il]
- [ce.sharif.ac.ir]
- [www.ic.unicamp.br]
- [www.ic.unicamp.br]
- [www.wisdom.weizmann.ac.il]
- [www.wisdom.weizmann.ac.il]
- [www.wisdom.weizmann.ac.il]
- [www.wisdom.weizmann.ac.il]
- DBLP

### Other Repositories/Bibliography

Venue: | ACM TRANSACTIONS ON ALGORITHMS |

Citations: | 150 - 8 self |

### BibTeX

@ARTICLE{Naor07novelarchitectures,

author = {Moni Naor and Udi Wieder},

title = {Novel Architectures for P2P Applications: the Continuous-Discrete Approach},

journal = {ACM TRANSACTIONS ON ALGORITHMS},

year = {2007},

volume = {3},

number = {3},

pages = {50--59}

}

### OpenURL

### Abstract

We propose a new approach for constructing P2P networks based on a dynamic decomposition of a continuous space into cells corresponding to processors. We demonstrate the power of these design rules by suggesting two new architectures, one for DHT (Distributed Hash Table) and the other for dynamic expander networks. The DHT network, which we call Distance Halving, allows logarithmic routing and load, while preserving constant degrees. Our second construction builds a network that is guaranteed to be an expander. The resulting topologies are simple to maintain and implement. Their simplicity makes it easy to modify and add protocols. We show it is possible to reduce the dilation and the load of the DHT with a small increase of the degree. We present a provably good protocol for relieving hot spots and a construction with high fault tolerance. Finally we show that, using our approach, it is possible to construct any family of constant degree graphs in a dynamic environment, though with worst parameters. Therefore we expect that more distributed data structures could be designed and implemented in a dynamic environment.

### Citations

3793 | Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications
- Stoica, Morris, et al.
- 2001
(Show Context)
Citation Context ...le sharing protocols. Recently a new and powerful generation of scalable overlay networks were proposed, that support a distributed hash table (DHT) functionality. Among them are Tapestry [24], Chord =-=[21]-=-, Pastry [19], CAN [18] and Viceroy [14]. In these systems data items are associated with a key and each processor in the system is responsible for storing a certain range of keys. These systems suppo... |

2917 | A Scalable Content-Addressable Network
- Ratnasamy, Francis, et al.
- 2001
(Show Context)
Citation Context ...ecently a new and powerful generation of scalable overlay networks were proposed, that support a distributed hash table (DHT) functionality. Among them are Tapestry [24], Chord [21], Pastry [19], CAN =-=[18]-=- and Viceroy [14]. In these systems data items are associated with a key and each processor in the system is responsible for storing a certain range of keys. These systems support a routing protocol t... |

1707 | The Probabilistic Method
- Alon, Spencer
- 2000
(Show Context)
Citation Context ...system is synchronized, this assumption is for convenience and does not play a major role. is q r . We have: Pr[X ] Pr X (1 + r q ) q r : Dene = r q . We have by Cherno's bound (c.f.[2] P.268): Pr h X (1 + ) q r i e log q r 1 n 2 : Applying the union bound yields that w.h.p all processors of layer l received less than requests in the epoch, therefore by the end of the e... |

1574 | Pastry: Scalable, Decentralized Object Location and Routing for Large-Scale Peer-toPeer Systems - Rowstron, Druschel - 2001 |

1364 |
Introduction to Parallel Algorithms and Architectures: Arrays
- Leighton
- 1992
(Show Context)
Citation Context ...s and 2 r+1 directed edges. Each node corresponding to an r bit binary string. There is a directed edge from each node u 1 u 2 u r to nodes u 2 u r 1 0 and u 2 u r 1 1. See Leighton [11] for an overview of various properties of this graph. The Distance Halving DHT emulates the De-Bruijn graph in the following sense. Assume that n = 2 r . Let ~x be a set of m points such that x i = i ... |

1118 | Tapestry: An Infrastructure for Fault-Tolerant WideArea Location and Routing
- Zhuang, Zhao, et al.
- 2001
(Show Context)
Citation Context ...net based file sharing protocols. Recently a new and powerful generation of scalable overlay networks were proposed, that support a distributed hash table (DHT) functionality. Among them are Tapestry =-=[24]-=-, Chord [21], Pastry [19], CAN [18] and Viceroy [14]. In these systems data items are associated with a key and each processor in the system is responsible for storing a certain range of keys. These s... |

671 | The small-world phenomenon: An algorithmic perspective
- Kleinberg
- 2000
(Show Context)
Citation Context ... by designing and proving the algorithms (e.g. 1 Lookup Scheme dilation congestion linkage Chord [21] log n (log n)=n log n Tapestry [24] log n (log n)=n log n CAN [18] dn 1=d dn 1=d 1 d Small Worlds =-=[-=-9] log 2n (log 2n)=n O(1) Viceroy [14] log n (log n)=n O(1) Distance Halving (ours) log d n (2 d p n) (log d n)=n O(d) Table 1: Comparison of expected performance measures of lookup schemes. for rou... |

620 | Kademlia: A peer-to-peer information system based on the xor metric
- Maymounkov, Mazieres
- 2002
(Show Context)
Citation Context ... it received a considerable amount of attention from the research community. Previous DHT designs include the Plaxton-Rajaraman design [39], Tapestry [48], Chord [45], Pastry [43], CAN [41], Kademlia =-=[32]-=-, Viceroy [29] and many more. These systems follow the general paradigm of consistent hashing [19]: Let I ∗ Preliminary versions of the paper appeared in [36] and [35]. † Research supported in part by... |

576 | Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web, STOC
- Karger, Lehman, et al.
- 1997
(Show Context)
Citation Context ...structing scalable networks. We rely on constructions over a continuous space in order to build discrete protocols. Previous works (such as Chord [21], Viceroy, [14], CAN [18], and consistent hashing =-=[8]-=-) have used continuous analogues to construct discrete structures as well. We differ by designing and proving the algorithms (e.g. 1 Lookup Scheme dilation congestion linkage Chord [21] log n (log n)=... |

471 | A hierarchical Internet object cache
- Chankhunthod, Danzig, et al.
- 1996
(Show Context)
Citation Context ...ata item to other processors, such that the load of handling all requests is distributed between a large number of processors. Various replication techniques were suggested in the literature ([8],[4],=-=[3]-=-), most notedly Karger et al [8] suggest a caching capable of relieving hot spots in a large dynamic network. Their protocol makes a distinction between cache servers and users while ours makes no suc... |

422 | Byzantine quorum systems
- Malkhi, Reiter
- 1998
(Show Context)
Citation Context ...lgorithm depends only on the expansion of the network. If the network is an expander then the algorithm balances the load in O(log n) rounds. Probabilistic quorums were suggested by Malkhi et al in [=-=17]-=-. A quorum set is chosen by randomly sampling p n processors. A random walk on an expander could serve as a procedure for sampling p n processors in a dynamic setting. 6. EMULATING GENERAL GRAPHS In t... |

405 | A digital fountain approach to reliable distribution of bulk data
- Byers, Luby, et al.
- 1998
(Show Context)
Citation Context ...associated with the same data item in parallel can be simple and efficient. It suggests storing the data using an erasure correcting code, (for instance the digital fountains suggested by Byers et al =-=[2]-=-) and thus avoid the need for replication. The data stored by any small subset of the processors would suffice to reconstruct the date item. Weatherspoon and Kubiatowicz [23] suggest that an erasure c... |

313 | Viceroy: A Scalable and Dynamic Emulation of the Butterfly
- Malkhi, Naor, et al.
- 2002
(Show Context)
Citation Context ... powerful generation of scalable overlay networks were proposed, that support a distributed hash table (DHT) functionality. Among them are Tapestry [24], Chord [21], Pastry [19], CAN [18] and Viceroy =-=[14]-=-. In these systems data items are associated with a key and each processor in the system is responsible for storing a certain range of keys. These systems support a routing protocol that allows users ... |

216 | A scheme for fast parallel communication - Valiant - 1982 |

211 | Analysis of the evolution of peer-to-peer systems
- Liben-Nowell, Balakrishnan, et al.
- 2002
(Show Context)
Citation Context ...of DH, Chord [21] and Viceroy [14]. It is possible to show (and would be shown in the full version of the paper) that G ~x is quite similar to chord. The various algorithms presented for Chord ([21], =-=[12]-=- ,[5]) apply here as well. 4 Achieving Smoothness Via our technique many problems reduce to the problem of achieving smoothness in I in a distributed manner. In this section we suggest various algorit... |

208 | Erasure coding vs. replication: A quantitative comparison
- Weatherspoon, Kubiatowicz
- 2002
(Show Context)
Citation Context ...s suggested by Byers et al [2]) and thus avoid the need for replication. The data stored by any small subset of the processors would suffice to reconstruct the date item. Weatherspoon and Kubiatowicz =-=[23]-=- suggest that an erasure correcting code may improve significantly the bandwidth and storage used by the system. 2 It may be that x i is chosen by hashing some i.d. of i. In this case it is important ... |

199 |
How to emulate shared memory
- Ranade
- 1991
(Show Context)
Citation Context ...ntly changes in the cached data item itself. Previous Work Various caching techniques were suggested in the literature, which operate under various distributed models (e.g. [39],[19],[8],[7]). Ranade =-=[40]-=- was the first to deal with the problem of hot-spots in routing (for simulating PRAMs) and showed that in the Butterfly network, which is closely related to the De-Bruijn network, it is possible to pr... |

188 | Koorde: A simple degree-optimal distributed hash table
- Kaashoek, Karger
- 2003
(Show Context)
Citation Context ...ired by the De-Bruijn graph. We are not the only ones to use the DeBruijn graph in this context. Constructions using it were suggested independently by Fraigniaud and Gauron [12], Kaashoek and Karger =-=[18]-=- and Abraham et al.[1]. The parameters they achieve are similar to those proved in Section 2, yet their approach in emulating the De-Bruijn graph is different. They try to do it directly whereas we us... |

187 | A.: Random walks in peer-to-peer networks
- Gkantsidis, Mihail, et al.
- 2004
(Show Context)
Citation Context ...in combinatorics. They have numerous applications in computer science. Applications in distributed computing include load balancing, fault tolerance and search through random walks (c.f. [2],[9],[27],=-=[15]-=-, [4]). It is well known that a random regular graph is an expander with high probability [13]. An explicit and deterministic construction for expanders was given by Margulis [31] and Gabber and Galil... |

161 | Simple efficient load balancing algorithms for peer-to-peer systems
- Karger, Ruhl
- 2004
(Show Context)
Citation Context ...hen servers join and leave the system. In practice most of the algorithms we show assume that some properties are behaving randomly. The problem of achieving smoothness was also addressed in [1],[30],=-=[20]-=-. In the following we always assume that I is continuous. All bounds remain correct even if points are perturbed by a polynomially small values, therefore a allocating 3 log n bits per variable is eno... |

139 | Routing algorithms for DHTs: Some open questions
- Ratnasamy, Shenker, et al.
- 2002
(Show Context)
Citation Context ...he occurrence of hotspots. Dynamic caching achieved a considerable amount of attention under many dierent models. The problem of dynamic caching in DHT's was specically raised by Ratnasamy et al at [2=-=4-=-]. To the best of our knowledge the algorithm we present is thesrst to ensure this property. In Section 4 we present several algorithms for maintaining a good load balancing between processors. These... |

102 | Building Peer-to-Peer Systems With Chord, a Distributed Lookup
- Dabek, Brunskill, et al.
- 2001
(Show Context)
Citation Context ... Chord [21] and Viceroy [14]. It is possible to show (and would be shown in the full version of the paper) that G ~x is quite similar to chord. The various algorithms presented for Chord ([21], [12] ,=-=[5]-=-) apply here as well. 4 Achieving Smoothness Via our technique many problems reduce to the problem of achieving smoothness in I in a distributed manner. In this section we suggest various algorithms f... |

102 | The power of two random choices: A survey of the techniques and results
- Mitzenmacher, Richa, et al.
- 2000
(Show Context)
Citation Context ... would remain logarithmic. Thus if we insist on ~x being smooth, then a balancing mechanism must be implemented. The idea of the following algorithm , (following the spirit of the two choice paradigm =-=[16]-=-), is to let a joining processor choose many locations, and set its x-value to be the best location found. Multiple Choice Join: 1. Estimate log n. 2. Sample t log n random points from (0; 1), when t ... |

102 |
Ashutosh Rege. A digital fountain approach to reliable distribution of bulk data
- Byers, Luby, et al.
- 1998
(Show Context)
Citation Context ...associated with the same data item in parallel can be simple and efficient. It suggests storing the data using an erasure correcting code, (for instance the digital fountains suggested by Byers et al.=-=[5]-=-) and thus avoid the need for replication. The data stored by any small subset of the servers suffices to reconstruct the data item. It is known that often the use of erasure correcting codes is more ... |

98 |
Explicit constructions of linear size superconcentrators
- GABBER, GALIL
- 1979
(Show Context)
Citation Context ... [4]). It is well known that a random regular graph is an expander with high probability [13]. An explicit and deterministic construction for expanders was given by Margulis [31] and Gabber and Galil =-=[14]-=-, and was later generalized by Cai [6]. The goal in this Section is to construct a P2P network which is guaranteed to be an expander. Independently from this paper, Law and Siu [25] used the fact that... |

96 | A proof of alon's second eigenvalue conjecture
- Friedman
- 2003
(Show Context)
Citation Context ...uted computing include load balancing, fault tolerance and search through random walks (c.f. [2],[9],[27],[15], [4]). It is well known that a random regular graph is an expander with high probability =-=[13]-=-. An explicit and deterministic construction for expanders was given by Margulis [31] and Gabber and Galil [14], and was later generalized by Cai [6]. The goal in this Section is to construct a P2P ne... |

93 |
Koorde: A simple degree-optimal hash table
- Kaashoek, Karger
- 2003
(Show Context)
Citation Context ...by the De-Bruijn graph. We are not the only ones to use the De-Bruijn graph in this context. Constructions using it were suggested independently by Fraigniaud and Gauron [6] and by Kaahoek and Karger =-=[8-=-]. The parameters they achieve are similar to those proved in Section 2, yet their approach is completely dierent. The construction is verysexible as well. In [21] we show how a slight variation of it... |

86 |
Explicit constructions of linear-sized superconcentrators
- Gabber, Galil
- 1981
(Show Context)
Citation Context ...twork. Our framework could be used to guarantee that the topology of the network is indeed an expander. An explicit construction for expanders was given by Margulis [15] and later by Gabber and Galil =-=[7]-=-. In this construction a continuous graph G is defined over I by the following two transformations: f(x; y) = (x + y; y) mod 1 , g(x; y) = (x; x + y) mod 1. Theorem 5.1 ([7]). For every set A of point... |

83 | Distributed Construction of Random Expander Networks
- Law, Sui
- 2003
(Show Context)
Citation Context ...decomposition of I into cells. An important parameter of the decomposition of I is the ratio 1 Law and Siu have independently designed another algorithm which builds an expander with high probability =-=[12-=-]. Their approach is completely dierent than ours. l(x) r(x) 0 1 x 0 (x+1)/2 1 x l(x) r(x) x/2 2x Figure 1: The uppersgure demonstrates the edges of a point in the continuous graph. The lowersgure sho... |

54 |
Explicit Construction of Concentrators
- Margulis
- 1973
(Show Context)
Citation Context ...nds only on the expansion of the network. Our framework could be used to guarantee that the topology of the network is indeed an expander. An explicit construction for expanders was given by Margulis =-=[15]-=- and later by Gabber and Galil [7]. In this construction a continuous graph G is defined over I by the following two transformations: f(x; y) = (x + y; y) mod 1 , g(x; y) = (x; x + y) mod 1. Theorem 5... |

54 | Dynamically fault-tolerant content addressable networks
- Saia, Fiat, et al.
- 2002
(Show Context)
Citation Context ...he Chord system [21] is resilient against random faults in the fail-stop model. It does not seem likely that Chord can be made spam resistant without a significant change in its design. Fiat and Saia =-=[20]-=- propose a content addressable network that is robust against deletion and spam in the worst case scenario, i.e. when an adversary can choose which processors fail. Clearly in this model some small fr... |

54 | A generic scheme for building overlay networks in adversarial scenarios
- Abraham, Awerbuch, et al.
- 2003
(Show Context)
Citation Context ...raph. We are not the only ones to use the DeBruijn graph in this context. Constructions using it were suggested independently by Fraigniaud and Gauron [12], Kaashoek and Karger [18] and Abraham et al.=-=[1]-=-. The parameters they achieve are similar to those proved in Section 2, yet their approach in emulating the De-Bruijn graph is different. They try to do it directly whereas we use the continuous-discr... |

50 | The Content-Addressable Network D2B
- Fraigniaud, Gauron
- 2003
(Show Context)
Citation Context ... DHT construction is inspired by the De-Bruijn graph. We are not the only ones to use the De-Bruijn graph in this context. Constructions using it were suggested independently by Fraigniaud and Gauron =-=[6-=-] and by Kaahoek and Karger [8]. The parameters they achieve are similar to those proved in Section 2, yet their approach is completely dierent. The construction is verysexible as well. In [21] we sho... |

49 | A simple fault tolerant distributed hash table
- Naor, Wieder
- 2003
(Show Context)
Citation Context ... Gauron [6] and by Kaahoek and Karger [8]. The parameters they achieve are similar to those proved in Section 2, yet their approach is completely dierent. The construction is verysexible as well. In [=-=21-=-] we show how a slight variation of it is robust against random faults of the processors. In Section 3 we show a dynamic caching algorithm that provably ensures that under any set of requests for dat... |

47 | Balanced binary trees for id management and load balance in distributed hash tables - Manku - 2004 |

45 | Asymptotically Efficient Approaches to Fault-Tolerance in Peer-to-peer Networks
- Hildrum, Kubiatowicz
- 2004
(Show Context)
Citation Context ...tems are known to be robust under random deletions ([48], [45], [41]). Stoica et al.prove that the Chord system [45] is resilient against random faults in the fail-stop model, Hildrum and Kubiatowicz =-=[16]-=- proved the resilience of Pastry and Tapestry. It does not seem likely that these systems could be made resistant under false message injection without a significant change in their design. Fiat et al... |

45 | Explicit Constructions of Concentrators,” Problems of Information Transactions - Margulis - 1973 |

44 | Workpreserving emulations of fixed-connection networks
- Koch, Leighton, et al.
- 1989
(Show Context)
Citation Context ... d. In other words if ~x is smooth then G ~x is a real time emulation of G l . Particularly this means any computation performed by G l , could be performed by G ~x in constant slow down. see ([13], [=-=10]-=-) for an overview on the literature of real time emulations. It is important to notice that assuming all processors know what n is, each processor can calculate separately which are its neighbors. Nex... |

43 | Censorship Resistant Peer-to-Peer Content Addressable Networks - Fiat, Saia - 2002 |

42 | BApproximate load balancing on dynamic and asynchronous networks
- Aiello, Awerbuch, et al.
- 1993
(Show Context)
Citation Context ...ons in computer science in general and distributed computing in particular. One such application is online load balancing jobs. An algorithm for online job load balancing in a network is presented in =-=[1]-=-. The efficiency of the algorithm depends only on the expansion of the network. Our framework could be used to guarantee that the topology of the network is indeed an expander. An explicit constructio... |

42 | Estimating network size from local information
- Horowitz, Malkhi
(Show Context)
Citation Context ...and the cost is not more than O(ρ) per change. Next we remove the assumption that all servers know the value of n, using the smoothness (there are other methods for doing so, as discussed in [29] and =-=[17]-=-). A smooth �x implies that each server Vi can estimate the value of n setting ni = 1 |s(Vi)| (ni is server Si’s guess to n). By definition of smoothness maxi,j ni nj = ρ(�x). Therefore it holds that ... |

40 |
Expanders might be practical: Fast algorithms for routing around faults in a multibutterfly
- Leighton, Maggs
- 1989
(Show Context)
Citation Context ...ures in combinatorics. They have numerous applications in computer science. Applications in distributed computing include load balancing, fault tolerance and search through random walks (c.f. [2],[9],=-=[27]-=-,[15], [4]). It is well known that a random regular graph is an expander with high probability [13]. An explicit and deterministic construction for expanders was given by Margulis [31] and Gabber and ... |

35 |
resistant peer-to-peer content addressable networks
- Fiat, Saia
- 2002
(Show Context)
Citation Context ...complicated and involves a heavy use of randomization. We feel that randomization offers a convenient opportunity for an adversary to diverge from the designated protocol. In a later paper Fiat et al =-=[6]-=- solve the first problem yet they do not describe a spam resistant lookup. 6.2 The Overlapping Distance Halving DHT Our construction (yet again) is a discretization of a continuous graph. The continuo... |

29 | Scalable and dynamic quorum systems
- Naor, Wieder
(Show Context)
Citation Context ...the smoothness of the construction. Techniques similar in spirit to those of Section 4 may guarantee smoothness. The relation between variants of the Join algorithm and the smoothness is discussed in =-=[20-=-]. 5.2 Constructing Expanders Expander graphs are graphs that are very `well connected' in the sense that for every set of vertices S of size at most 1 2 jV j there are at least jSj vertices in V nS t... |

18 |
Pastry: Scalable, decentraized object location and routing for large-scale peerto-peer systems
- Rowstron, Druschel
- 2001
(Show Context)
Citation Context ...otocols. Recently a new and powerful generation of scalable overlay networks were proposed, that support a distributed hash table (DHT) functionality. Among them are Tapestry [28], Chord [26], Pastry =-=[25]-=-, CAN [23] and Viceroy [16]. In these systems data items are associated with a key and each processor in the system is responsible for storing a certain range of keys. These systems support a routing ... |

17 | Concentration of measure for the analysis of randomised algorithms - Panconesi, Dubhashi |

15 | Decentralized algorithms using both local and random probes for p2p load balancing - Kenthapadi, Manku - 2005 |

13 |
Dubhashi and Alessandro Panconesi. Concentration of Measure for the Analysis of Randomized Algorithms
- Devdatt
- 2009
(Show Context)
Citation Context ... by V in the second phase of the routing. The analysis of Theorem 2.10 holds in this case as well so we know E[ � Xi] is O(log n) and that the Xis are (log n)-wise independent. We use Theorem 4.21 in =-=[10]-=- which is a high moment version of Chebyshev’s inequality. 5 Section 3 deals with the case where the same item is queried by multiple servers. 11sClaim 2.12. Let X1, X2, . . . , Xn be random variables... |

12 |
Viceroy: A scalable and dynamic emulation of the butter
- Malkhi, Naor, et al.
- 2002
(Show Context)
Citation Context ... powerful generation of scalable overlay networks were proposed, that support a distributed hash table (DHT) functionality. Among them are Tapestry [28], Chord [26], Pastry [25], CAN [23] and Viceroy =-=[16]-=-. In these systems data items are associated with a key and each processor in the system is responsible for storing a certain range of keys. These systems support a routing protocol that allows users ... |

10 | Fast faulttolerant concurrent access to shared objects - Plaxton, Rajaraman - 1996 |