## Decentralized algorithms using both local and random probes for p2p load balancing (2005)

### Cached

### Download Links

- [theory.stanford.edu]
- [www-db.stanford.edu]
- [infolab.stanford.edu]
- [www.ic.unicamp.br]
- DBLP

### Other Repositories/Bibliography

Venue: | In Seventeenth ACM Symposium on Parallelism in Algorithms and Architectures (SPAA |

Citations: | 12 - 0 self |

### BibTeX

@INPROCEEDINGS{Kenthapadi05decentralizedalgorithms,

author = {Krishnaram Kenthapadi and Gurmeet Singh Manku},

title = {Decentralized algorithms using both local and random probes for p2p load balancing},

booktitle = {In Seventeenth ACM Symposium on Parallelism in Algorithms and Architectures (SPAA},

year = {2005},

pages = {135--144}

}

### OpenURL

### Abstract

We study randomized algorithms for placing a sequence of n nodes on a circle with unit perimeter. Nodes divide the circle into disjoint arcs. We desire that a newly-arrived node (which is oblivious of its index in the sequence) choose its position on the circle by learning the positions of as few existing nodes as possible. At the same time, we desire that that the variation in arc-lengths be small. To this end, we propose a new algorithm that works as follows: The k th node chooses r random points on the circle, inspects the sizes of v arcs in the vicinity of each random point, and places itself at the mid-point of the largest arc encountered. We show that for any combination of r and v satisfying rv ≥ c log k, where c is a small constant, the ratio of the largest to the smallest arc-length is at most eight w.h.p., for an arbitrarily long

### Citations

3460 | Chord: A scalable peer-to-peer lookup service for internet applications
- Stoica, Morris, et al.
- 2001
(Show Context)
Citation Context ...a function of their IDs. These connections are of two kinds: short-distance links and long-distance. The shortdistance links connect nodes with adjacent IDs in [0, 1), thereby forming a ring 2 (Chord =-=[45]-=-, D2B [12], Koorde [23], SkipNet [20], Symphony [34], Viceroy [30], Ulysses [28], and others [1,31,40]). Some DHT routing networks do not form a complete ring (examples: Pastry [44], Tapestry [47], Ba... |

2689 | A scalable content addressable network
- Ratnasamy, Francis, et al.
- 2001
(Show Context)
Citation Context ...ation in partition sizes for load balance among managers. 1.4 Load Balance and ID Assignment in DHTs Early DHT designs allowed each host to independently choose a number in [0, 1) uniformly at random =-=[12, 20, 23, 30, 34, 42, 44, 45, 47]-=-. Such an algorithm is decentralized and requires zero messages to select an ID. However, σ = Ω(log 2 n) [40] w.h.p. King and Saia [26] recently established that σ = Θ(n log n) w.h.p. If each host cho... |

1493 | Probability inequalities for sums of bounded random variables
- Hoeffding
- 1963
(Show Context)
Citation Context .... Else, we select one of the non-full bins (from among the d choices) at random and place a ball into it. The first lemma below contains two useful forms of inequalities by Chernoff [9] and Hoeffding =-=[21]-=-. The second lemma helps us derive tail bounds for dependent binary random variables under certain conditions. Lemma 3.1. Let Z denote a random variable with a binomial distribution Z ∼ B(n, p). For e... |

1416 | P: Pastry: scalable, decentralized object location, and routing for large-scale peer-to-peer systems
- Rowstron, Druschel
- 2001
(Show Context)
Citation Context ...ng a ring 2 (Chord [45], D2B [12], Koorde [23], SkipNet [20], Symphony [34], Viceroy [30], Ulysses [28], and others [1,31,40]). Some DHT routing networks do not form a complete ring (examples: Pastry =-=[44]-=-, Tapestry [47], Bamboo [43] and Kademlia [36]). However, almost all of the ring connections do exist in these networks because nodes in the circular ID space [0, 1) are divided into disjoint clusters... |

845 | I.: Wide-area cooperative storage with cfs
- Dabek, Kaashoek, et al.
- 2001
(Show Context)
Citation Context ...ost chooses a random number in [0, 1) and splits the partition the number falls into, σ diminishes to Θ(log n) [2, 40]. Further improvement is possible by assigning multiple virtual IDs per host (CFS =-=[10]-=- uses this idea). With Θ(log n) IDs per host, σ = Θ(1) (see the consistent hashing paper [24], for example). However, by using virtual IDs, the number of overlay connections per host gets amplified by... |

714 |
A Measure of Asymptotic Efficiency of Tests for a Hypothesis Based on a Sum of Observations
- Chernoff
- 1952
(Show Context)
Citation Context ... (the trial fails). Else, we select one of the non-full bins (from among the d choices) at random and place a ball into it. The first lemma below contains two useful forms of inequalities by Chernoff =-=[9]-=- and Hoeffding [21]. The second lemma helps us derive tail bounds for dependent binary random variables under certain conditions. Lemma 3.1. Let Z denote a random variable with a binomial distribution... |

554 | Kademlia: A peer-to-peer information system based on the xor metric
- Maymounkov, Mazieres
- 2002
(Show Context)
Citation Context ..., SkipNet [20], Symphony [34], Viceroy [30], Ulysses [28], and others [1,31,40]). Some DHT routing networks do not form a complete ring (examples: Pastry [44], Tapestry [47], Bamboo [43] and Kademlia =-=[36]-=-). However, almost all of the ring connections do exist in these networks because nodes in the circular ID space [0, 1) are divided into disjoint clusters 3 and nodes within a cluster form a clique. A... |

523 | Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the world wide web
- Karger, Lehman, et al.
- 1997
(Show Context)
Citation Context ...ishes to Θ(log n) [2, 40]. Further improvement is possible by assigning multiple virtual IDs per host (CFS [10] uses this idea). With Θ(log n) IDs per host, σ = Θ(1) (see the consistent hashing paper =-=[24]-=-, for example). However, by using virtual IDs, the number of overlay connections per host gets amplified by a factor of Ω(log n) – this is costly because higher degree overlay networks require more re... |

438 | Tapestry: A resilient global-scale overlay for service development
- Zhao, Huang, et al.
(Show Context)
Citation Context ...ord [45], D2B [12], Koorde [23], SkipNet [20], Symphony [34], Viceroy [30], Ulysses [28], and others [1,31,40]). Some DHT routing networks do not form a complete ring (examples: Pastry [44], Tapestry =-=[47]-=-, Bamboo [43] and Kademlia [36]). However, almost all of the ring connections do exist in these networks because nodes in the circular ID space [0, 1) are divided into disjoint clusters 3 and nodes wi... |

337 | Handling churn in a dht
- Rhea, Geels, et al.
(Show Context)
Citation Context ... [12], Koorde [23], SkipNet [20], Symphony [34], Viceroy [30], Ulysses [28], and others [1,31,40]). Some DHT routing networks do not form a complete ring (examples: Pastry [44], Tapestry [47], Bamboo =-=[43]-=- and Kademlia [36]). However, almost all of the ring connections do exist in these networks because nodes in the circular ID space [0, 1) are divided into disjoint clusters 3 and nodes within a cluste... |

295 | Viceroy: A scalable and dynamic emulation of the butterfly
- Malkhi, Naor, et al.
- 2002
(Show Context)
Citation Context ...-distance links and long-distance. The shortdistance links connect nodes with adjacent IDs in [0, 1), thereby forming a ring 2 (Chord [45], D2B [12], Koorde [23], SkipNet [20], Symphony [34], Viceroy =-=[30]-=-, Ulysses [28], and others [1,31,40]). Some DHT routing networks do not form a complete ring (examples: Pastry [44], Tapestry [47], Bamboo [43] and Kademlia [36]). However, almost all of the ring conn... |

282 | Skipnet: A scalable overlay network with practical locality properties
- Harvey, Jones, et al.
- 2003
(Show Context)
Citation Context ...tions are of two kinds: short-distance links and long-distance. The shortdistance links connect nodes with adjacent IDs in [0, 1), thereby forming a ring 2 (Chord [45], D2B [12], Koorde [23], SkipNet =-=[20]-=-, Symphony [34], Viceroy [30], Ulysses [28], and others [1,31,40]). Some DHT routing networks do not form a complete ring (examples: Pastry [44], Tapestry [47], Bamboo [43] and Kademlia [36]). However... |

249 | Balanced allocations
- Azar, Broder, et al.
- 1999
(Show Context)
Citation Context ...h n bins and n balls, the height of the fullest bin is Θ(log n/ log log n) (see Gonnet [18], Mitzenmacher [37] and Raab and Steger [41]). For the case d ≥ 2, a breakthrough was achieved by Azar et al =-=[6]-=- who showed that the height of the fullest bin is log log n/ log d+Θ(1), if the least-loaded bin among the d bins is chosen at each trial. For further results and a survey of proof techniques for d ≥ ... |

246 | The impact of DHT routing geometry on resilience and proximity
- Gummadi, Gummadi, et al.
(Show Context)
Citation Context ...s based on hypercubes, R = Θ(log n) when each node makes Θ(log n) links per node (examples: Chord [14, 45], Pastry [44] and Tapestry [47]). Concerns for “physical networkproximity” (see Gummadi et al =-=[19]-=- for a discussion of this issue) have resulted in the creation of randomized vari2 In practice, a host makes connection with f hosts adjacent to it along the circle for a fault-tolerant ring, where f ... |

237 | Skip graphs
- Aspnes, Shah
(Show Context)
Citation Context ...e randomized butterflies [31], and several randomized networks analyzed by Manku, Naor and Wieder [35] — these include randomized-Chord [19, 46], randomizedhypercubes [19], Symphony [34], skip-graphs =-=[5]-=- and SkipNet [20]. The ID Assignment Problem: Upon arrival, a new host has to be assigned an ID. It is customarily assumed in DHT design that the new host “knows” one existing member of the ring at th... |

199 | The power of two choices in randomized load balancing
- Mitzenmacher
- 2001
(Show Context)
Citation Context ...een interest in the computer science community in analyzing the height of the fullest bin. With n bins and n balls, the height of the fullest bin is Θ(log n/ log log n) (see Gonnet [18], Mitzenmacher =-=[37]-=- and Raab and Steger [41]). For the case d ≥ 2, a breakthrough was achieved by Azar et al [6] who showed that the height of the fullest bin is log log n/ log d+Θ(1), if the least-loaded bin among the ... |

169 | Symphony: distributed hashing in a small world
- Manku, Bawa, et al.
(Show Context)
Citation Context ...o kinds: short-distance links and long-distance. The shortdistance links connect nodes with adjacent IDs in [0, 1), thereby forming a ring 2 (Chord [45], D2B [12], Koorde [23], SkipNet [20], Symphony =-=[34]-=-, Viceroy [30], Ulysses [28], and others [1,31,40]). Some DHT routing networks do not form a complete ring (examples: Pastry [44], Tapestry [47], Bamboo [43] and Kademlia [36]). However, almost all of... |

143 | Novel architectures for P2P applications: The continuous-discrete approach
- Naor, Wieder
(Show Context)
Citation Context ... Three previous results can be stated in terms of local and random probes. With one random probe, followed by splitting the arc encountered by the random probe, σ = Θ(log n) for a sequence of n nodes =-=[2, 40]-=-. Naor and Wieder [40] and Abraham et al [1] showed that with O(log n) random probes (followed by no local probes), σ = Θ(1) (it is assumed that the i th node knows n, the length of the sequence). Man... |

141 | Simple efficient load balancing algorithms for peer-to-peer systems
- Karger, Ruhl
(Show Context)
Citation Context ...n) R No Adler et al [2] Θ(1) Θ(R + log n) No Only for hypercubic routing networks. Naor & Wieder [40] Θ(1) Θ(R log n) (Yes) No proof for departures. Abraham et al [1] Θ(1) Θ(R log n) No Karger & Ruhl =-=[25]-=- Θ(1) Θ(R log n) Yes O(log log n) re-assignments per arrival/departure. Giakkoupis & Hadzilacos [15] 4 Θ(R log n) Yes 1 re-assignment per departure. Manku [32] 4 Θ(R + log n) Yes 1 re-assignment per d... |

105 | Graph-Theoretic Analysis of Structured Peer-to-Peer Systems: Routing Distances and Fault Resilience
- Loguinov, Casas, et al.
- 2005
(Show Context)
Citation Context ...rs” are used for greedy routing [35]. In fact, R = Θ(log n/ log log n) holds for a variety of networks with Θ(log n) links per node: high-degree de Bruijn networks (an observation made by many groups =-=[1, 12, 23, 29, 40]-=-), high-degree butterflies [28], Kleinberg-style randomized butterflies [31], and several randomized networks analyzed by Manku, Naor and Wieder [35] — these include randomized-Chord [19, 46], randomi... |

99 | The power of two random choices: A survey of techniques and results
- Mitzenmacher, Richa, et al.
- 2000
(Show Context)
Citation Context ...t of the fullest bin is log log n/ log d+Θ(1), if the least-loaded bin among the d bins is chosen at each trial. For further results and a survey of proof techniques for d ≥ 2, see Mitzenmacher et al =-=[38]-=-. Our focus is on cliques, which are equivalent to bins with finite capacity. Moreover, we wish to bound the height of the bin with the fewest balls. Structured coupon collection over cliques was moti... |

95 | Online balancing of range-partitioned data with applications to peer-to-peer systems
- GANESAN, BAWA, et al.
(Show Context)
Citation Context ...ent blocks are fused into one block while a large block is split into two. The cost associated with a fusion or a split is proportional to the number of data elements that have to move. Ganesan et al =-=[13]-=- present an efficient deterministic algorithm that guarantees a constant number of moves per data element, amortized over the lifetime of the data structure. If each block is assigned to one machine i... |

89 |
Koorde: A Simple Degree-optimal Hash Table
- Kaashoek, Karger
(Show Context)
Citation Context .... These connections are of two kinds: short-distance links and long-distance. The shortdistance links connect nodes with adjacent IDs in [0, 1), thereby forming a ring 2 (Chord [45], D2B [12], Koorde =-=[23]-=-, SkipNet [20], Symphony [34], Viceroy [30], Ulysses [28], and others [1,31,40]). Some DHT routing networks do not form a complete ring (examples: Pastry [44], Tapestry [47], Bamboo [43] and Kademlia ... |

89 | Know thy neighbor’s neighbor: The power of lookahead in randomized p2p networks
- Manku, Naor, et al.
- 2004
(Show Context)
Citation Context ...ted size 16. ants of these networks — randomized-Chord [19, 46] and randomized-hypercubes [8, 19]. For these networks, R = Θ(log n/ log log n) when “neighbor-of-neighbors” are used for greedy routing =-=[35]-=-. In fact, R = Θ(log n/ log log n) holds for a variety of networks with Θ(log n) links per node: high-degree de Bruijn networks (an observation made by many groups [1, 12, 23, 29, 40]), high-degree bu... |

84 | Load balancing in dynamic structured p2p systems
- GODFREY, LAKSHMINARAYANAN, et al.
- 2004
(Show Context)
Citation Context ...locations (determined by multiple hash-values of the object-name). A drawback of this idea is the overhead associated with multiple probes necessary when storing and retrieving objects. Godfrey et al =-=[16]-=- take a systems approach, identifying the run-time loads on various nodes. They propose heuristics for re-distributing objects between pairs of lightly- and heavily-loaded nodes. Load balance for rang... |

80 |
Urn model and their applications: an approach. to Modern discrete probability theory
- Johnson, Kotz
- 1977
(Show Context)
Citation Context ... select one of the non-full bins (from among the d choices) at random and place a ball into it. The classic balls-and-bins problem involves bins with infinite capacity and d = 1 (see Johnson and Kotz =-=[22]-=- or Kolchin et al [27]). Recently, there has been interest in the computer science community in analyzing the height of the fullest bin. With n bins and n balls, the height of the fullest bin is Θ(log... |

78 |
Expected length of the longest probe sequence in hash code searching
- Gonnet
- 1981
(Show Context)
Citation Context ...cently, there has been interest in the computer science community in analyzing the height of the fullest bin. With n bins and n balls, the height of the fullest bin is Θ(log n/ log log n) (see Gonnet =-=[18]-=-, Mitzenmacher [37] and Raab and Steger [41]). For the case d ≥ 2, a breakthrough was achieved by Azar et al [6] who showed that the height of the fullest bin is log log n/ log d+Θ(1), if the least-lo... |

77 | Balls into bins” — a simple and tight analysis - Raab, Steger - 1998 |

67 | Problems and results in extremal combinatorics - Alon - 2003 |

52 | A generic scheme for building overlay networks in adversarial scenarios
- Abraham, Awerbuch, et al.
(Show Context)
Citation Context ...f local and random probes. With one random probe, followed by splitting the arc encountered by the random probe, σ = Θ(log n) for a sequence of n nodes [2, 40]. Naor and Wieder [40] and Abraham et al =-=[1]-=- showed that with O(log n) random probes (followed by no local probes), σ = Θ(1) (it is assumed that the i th node knows n, the length of the sequence). Manku [32] showed that with one random probe, f... |

45 | Routing networks for distributed hash tables
- Manku
(Show Context)
Citation Context ...ce. The shortdistance links connect nodes with adjacent IDs in [0, 1), thereby forming a ring 2 (Chord [45], D2B [12], Koorde [23], SkipNet [20], Symphony [34], Viceroy [30], Ulysses [28], and others =-=[1,31,40]-=-). Some DHT routing networks do not form a complete ring (examples: Pastry [44], Tapestry [47], Bamboo [43] and Kademlia [36]). However, almost all of the ring connections do exist in these networks b... |

45 | Balanced binary trees for ID management and load balance in distributed hash tables
- Manku
(Show Context)
Citation Context ...aor and Wieder [40] and Abraham et al [1] showed that with O(log n) random probes (followed by no local probes), σ = Θ(1) (it is assumed that the i th node knows n, the length of the sequence). Manku =-=[32]-=- showed that with one random probe, followed by a local probe of size O(log i) (where log i is estimated), σ ≤ 4. In contrast, our result shows that there is a smooth tradeoff between random and local... |

41 | Incrementally Improving Lookup Latency in Distributed Hash Table Systems
- Zhang, Goel, et al.
- 2003
(Show Context)
Citation Context ...e). We will ignore this issue, however, as it does not affect the asymptotes of our algorithms. 3 In Pastry, these are called leafsets with expected size 16. ants of these networks — randomized-Chord =-=[19, 46]-=- and randomized-hypercubes [8, 19]. For these networks, R = Θ(log n/ log log n) when “neighbor-of-neighbors” are used for greedy routing [35]. In fact, R = Θ(log n/ log log n) holds for a variety of n... |

38 | Choosing a random peer - King, Saia - 2004 |

37 | Heterogeneity and Load Balance in Distributed Hash Tables
- Godfrey, Stoica
- 2005
(Show Context)
Citation Context ...the schemes outlined above is either Θ(log 2 n/ log log n) messages [1,25,40] or Θ(log n) messages [2, 32], for networks with R = Θ(log n/ log log n). For load balancing over heterogeneous nodes, see =-=[15, 17]-=-. 1.5 Other DHT Load Balancing Problems Byers et al [7] suggest a variant of the power-of-two choices paradigm (see Mitzenmacher et al [38] for a survey). In their scheme, nodes continue to choose ran... |

32 | Load-balancing and Locality in RangeQueriable Data Structures
- Aspnes, Kirsch, et al.
- 2004
(Show Context)
Citation Context ...llest) block can be efficiently discovered. It is not clear how this operation can be implemented in a decentralized fashion without causing congestion in a DHT. Karger and Ruhl [25] and Aspnes et al =-=[4]-=- present efficient congestionfree algorithms that match large blocks with small blocks. Their algorithms are randomized, decentralized and handle a dynamic number of blocks. 1.6 Summary of Results 1. ... |

32 | Geometric generalizations of the power of two choices
- Byers, Considine, et al.
- 2004
(Show Context)
Citation Context ...essages [1,25,40] or Θ(log n) messages [2, 32], for networks with R = Θ(log n/ log log n). For load balancing over heterogeneous nodes, see [15, 17]. 1.5 Other DHT Load Balancing Problems Byers et al =-=[7]-=- suggest a variant of the power-of-two choices paradigm (see Mitzenmacher et al [38] for a survey). In their scheme, nodes continue to choose random numbers in [0, 1) as their IDs. However, an object ... |

18 | Ulysses: A robust, low-diameter, low-latency peer-to-peer network,” European transaction on telecommunications
- Kumar, Merugu, et al.
- 2004
(Show Context)
Citation Context ...s and long-distance. The shortdistance links connect nodes with adjacent IDs in [0, 1), thereby forming a ring 2 (Chord [45], D2B [12], Koorde [23], SkipNet [20], Symphony [34], Viceroy [30], Ulysses =-=[28]-=-, and others [1,31,40]). Some DHT routing networks do not form a complete ring (examples: Pastry [44], Tapestry [47], Bamboo [43] and Kademlia [36]). However, almost all of the ring connections do exi... |

8 | Dipsea: A Modular Distributed Hash Table - Manku - 2004 |

6 |
brief announcement) An overview of the content-addressable network D2B
- Fraigniaud, Gauron
- 2003
(Show Context)
Citation Context ... of their IDs. These connections are of two kinds: short-distance links and long-distance. The shortdistance links connect nodes with adjacent IDs in [0, 1), thereby forming a ring 2 (Chord [45], D2B =-=[12]-=-, Koorde [23], SkipNet [20], Symphony [34], Viceroy [30], Ulysses [28], and others [1,31,40]). Some DHT routing networks do not form a complete ring (examples: Pastry [44], Tapestry [47], Bamboo [43] ... |

4 |
Rowstron A. ”Topology-aware routing in structured peer-to-peer overlay networks
- Castro, Druschel, et al.
- 2001
(Show Context)
Citation Context ...ver, as it does not affect the asymptotes of our algorithms. 3 In Pastry, these are called leafsets with expected size 16. ants of these networks — randomized-Chord [19, 46] and randomized-hypercubes =-=[8, 19]-=-. For these networks, R = Θ(log n/ log log n) when “neighbor-of-neighbors” are used for greedy routing [35]. In fact, R = Θ(log n/ log log n) holds for a variety of networks with Θ(log n) links per no... |

2 |
Vijay V Vazirani. A stochastic process on the hypercube with applications to peer-to-peer networks
- Adler, Halperin, et al.
- 2003
(Show Context)
Citation Context ... [40] w.h.p. King and Saia [26] recently established that σ = Θ(n log n) w.h.p. If each host chooses a random number in [0, 1) and splits the partition the number falls into, σ diminishes to Θ(log n) =-=[2, 40]-=-. Further improvement is possible by assigning multiple virtual IDs per host (CFS [10] uses this idea). With Θ(log n) IDs per host, σ = Θ(1) (see the consistent hashing paper [24], for example). Howev... |

1 |
A scheme for load balancing in hetergoneous distributed hash tables
- Giakkoupis, Hadzilacos
- 2005
(Show Context)
Citation Context ...] Θ(1) Θ(R log n) (Yes) No proof for departures. Abraham et al [1] Θ(1) Θ(R log n) No Karger & Ruhl [25] Θ(1) Θ(R log n) Yes O(log log n) re-assignments per arrival/departure. Giakkoupis & Hadzilacos =-=[15]-=- 4 Θ(R log n) Yes 1 re-assignment per departure. Manku [32] 4 Θ(R + log n) Yes 1 re-assignment per departure. 1 + ɛ, Θ(R + 1 ɛ2 log n) Yes O(1/ɛ) re-assignments per departure. ɛ ∈ (0, 1] Our algorithm... |