## Tight Bounds for Parallel Randomized Load Balancing (1992)

### Cached

### Download Links

Venue: | Computing Research Repository |

Citations: | 5 - 0 self |

### BibTeX

@ARTICLE{Lenzen92tightbounds,

author = {Christoph Lenzen},

title = {Tight Bounds for Parallel Randomized Load Balancing},

journal = {Computing Research Repository},

year = {1992}

}

### OpenURL

### Abstract

We explore the fundamental limits of distributed balls-intobins algorithms, i.e., algorithms where balls act in parallel, as separate agents. This problem was introduced by Adler et al., who showed that non-adaptive and symmetric algorithms cannot reliably perform better than a maximum bin load of Θ(loglogn/logloglogn) within the same number of rounds. We present an adaptive symmetric algorithm that achieves a bin load of two in log ∗ n + O(1) communication rounds using O(n) messages in total. Moreover, larger bin loads can be traded in for smaller time complexities. We prove a matching lower bound of (1−o(1))log ∗ n on the time complexity of symmetric algorithms that guarantee small bin loads at an asymptotically optimal message complexity of O(n). The essential preconditions of the proof are (i) a limit of O(n) on the total number of messages sent by the algorithm and (ii) anonymity of bins, i.e., the port numberings of balls are not globally consistent. In order to show that our technique yields indeed tight bounds, we provide for each assumption an algorithm violating it, in turn achieving a constant maximum bin load in constant time. As an application, we consider the following problem. Given a fully connected graph of n nodes, where each node needs to send and receive up to n messages, and in each round each node may send one message over each link, deliver all messages as quickly as possible to their destinations. We give a simple and robust algorithm of time complexity O(log ∗ n) for this task and provide a generalization to the case where all nodes initially hold arbitrary sets of messages. Completing the picture, we give a less practical, but asymptotically optimal algorithm terminating within O(1) rounds. All these bounds hold with high probability.

### Citations

249 | Balanced allocations
- Azar, Broder, et al.
- 1999
(Show Context)
Citation Context ...PRAM) on a distributed memory machine (DMM) with a factor O(loglognlog ∗ n) overhead; in essence, their result was a solution to balls-into-bins with maximum bin load of O(loglogn) w.h.p. Azar et al. =-=[3]-=- generalized their result by showing that if the balls choose sequentially from d ≥ 2 u.i.r. bins greedily the currently least loaded one, the maximum load is loglogn/logd+O(1) w.h.p. 3 They prove tha... |

199 | The power of two choices in randomized load balancing
- Mitzenmacher
- 2001
(Show Context)
Citation Context ...h.p. In addition, quite a few variations of the basic problem have been studied. Since resources often need to be assigned to dynamically arriving tasks, infinite processes have been considered (e.g. =-=[3, 8, 29, 30, 31, 38, 40]-=-). In [32] it is shown that, inthesequentialsetting, memorizinggoodchoicesfrom previousballs has similar impactas increasingthenumberof fresh random choices. Awerbuch et al. [2] studied arbitrary Lp n... |

169 |
Locality in distributed graph algorithms
- Linial
- 1992
(Show Context)
Citation Context ...ds, it arises from the assumption that bins are (initially) anonymous (cf. Problems 6.1 and 6.2), which fits a wide range of real-world systems. Like Linial in his seminal work on 3-coloring the ring =-=[23]-=-, we attain a lower bound of Ω(log ∗ n) on the time required to solve the task efficiently. This connection is more than superficial, as both bounds essentially arise from a symmetry breaking problem.... |

85 |
auf der Heide, Efficient PRAM Simulation on a Distributed Memory Machine
- Karp, Luby, et al.
- 1994
(Show Context)
Citation Context ...oach is Θ(logn/loglogn) with high probability (w.h.p.) 2 [10]. With the growing interest in parallel computing, since the beginningoftheninetiesthetopicreceivedincreasinglymore attention. Karp et al. =-=[17]-=- demonstrated for the first time that two random choices are superior to one. By combining two (possibly not fully independent) hashing functions, they simulated a parallel random access machine (PRAM... |

81 | How Asymmetry Helps Load Balancing
- Vöcking
- 2003
(Show Context)
Citation Context ...lutions for the whole range of parameters. Given the existing lower bounds, since then the only possibility for further improvement has been to search for nonadaptive or asymmetric algorithms. Vöcking=-=[40]-=- introduced the sequential “always-go-left” algorithm which employs asymmetric tie-breaking in order to improve the impact of the number of possible choices d from logarithmic to linear. Furthermore, ... |

80 | How Useful is Old Information
- Mitzenmacher
- 1997
(Show Context)
Citation Context ...h.p. In addition, quite a few variations of the basic problem have been studied. Since resources often need to be assigned to dynamically arriving tasks, infinite processes have been considered (e.g. =-=[3, 8, 29, 30, 31, 38, 40]-=-). In [32] it is shown that, inthesequentialsetting, memorizinggoodchoicesfrom previousballs has similar impactas increasingthenumberof fresh random choices. Awerbuch et al. [2] studied arbitrary Lp n... |

78 |
Expected length of the longest probe sequence in hash code searching
- Gonnet
- 1981
(Show Context)
Citation Context ...orks, with applications from scientific computing to overlay networks. 2. RELATED WORK Probably one of the earliest applications of randomized load balancing has been hashing. In this context, Gonnet =-=[15]-=- proved that when throwing n balls uniformly and independently at random (u.i.r.) into n bins, the fullest bin has load (1 + o(1))logn/loglogn in expectation. It is also common knowledge that the maxi... |

56 | Parallel randomized load balancing
- Adler, Chakrabarti, et al.
- 1995
(Show Context)
Citation Context ...s quickly while keeping the maximum bin load small? As in other areas where centralized control must be avoided (sometimes because it is impossible), the key to success is randomization. Adler et al. =-=[1]-=- devised parallel randomized algorithms for the problem whose running times and maximum bin loads are essentially doubly-logarithmic. They provide a lower bound which is asymptotically matching the up... |

55 | On the Analysis of Randomized Load Balancing Schemes
- Mitzenmacher
(Show Context)
Citation Context ...h.p. In addition, quite a few variations of the basic problem have been studied. Since resources often need to be assigned to dynamically arriving tasks, infinite processes have been considered (e.g. =-=[3, 8, 29, 30, 31, 38, 40]-=-). In [32] it is shown that, inthesequentialsetting, memorizinggoodchoicesfrom previousballs has similar impactas increasingthenumberof fresh random choices. Awerbuch et al. [2] studied arbitrary Lp n... |

54 | Approximate equilibria and ball fusion
- Koutsoupias, Mavronicolas, et al.
(Show Context)
Citation Context ...p norms instead of the maximum bin load (i.e., the L∞ norm) as quality measure, showing that the greedy strategy is p-competitive to an offline algorithm. Several works addressed weighted balls (e.g. =-=[6, 7, 20, 36, 39]-=-) in order to model tasks of varying resource consumption. The case of heterogeneous bins was examined as well [41]. In recent years, balls-into-bins has also been considered from a game theoretic poi... |

44 |
Converting High Probability into Nearly-Constant Time --- with Applications to Parallel Hashing
- Matias, Vishkin
- 1991
(Show Context)
Citation Context ...of view [5, 19]. Results related to ours have been discovered before for hashing problems. A number of publications presents algorithms with running times of O(log ∗ n) (or very close) in PRAM models =-=[4, 14, 27, 28]-=-. At the heart of these routines as well as our balls-into-bins solutions lies the idea to use an in each iteration exponentially growing share of the available resources to deal with the remaining ke... |

41 |
Balls and bins: a study in negative dependence. Random Structures and Algorithms
- Dubhashi, Ranjan
- 1998
(Show Context)
Citation Context ...bins, the fullest bin has load (1 + o(1))logn/loglogn in expectation. It is also common knowledge that the maximum bin load of this simple approach is Θ(logn/loglogn) with high probability (w.h.p.) 2 =-=[10]-=-. With the growing interest in parallel computing, since the beginningoftheninetiesthetopicreceivedincreasinglymore attention. Karp et al. [17] demonstrated for the first time that two random choices ... |

40 | Hundreds of impossibility results for distributed computing
- Fich, Ruppert
(Show Context)
Citation Context ... may merely decide on how to further separate elements that share the same image under the hash functions applied to them so far. Various other techniques for obtaining distributed lower bounds exist =-=[13, 26]-=-, however, they are not related to our work. If graph-based, the arguments are often purely informationtheoretic, inthesensethatsomeinformation mustbe exchanged over some bottleneck link or node in a ... |

39 | Near-tight Lower Bound on the Time Complexity of Distributed MST Construction
- Peleg, Rubinovich
(Show Context)
Citation Context ... the arguments are often purely informationtheoretic, inthesensethatsomeinformation mustbe exchanged over some bottleneck link or node in a carefully constructed network with diameter larger than two =-=[25, 35]-=-. In our setting, such information theoretic lower bounds will not work: Any two balls may exchange information along n edge-disjoint paths of length two, as the graph describing which edges could pot... |

35 | A hundred impossibility proofs for distributed computing
- Lynch
- 1989
(Show Context)
Citation Context ... may merely decide on how to further separate elements that share the same image under the hash functions applied to them so far. Various other techniques for obtaining distributed lower bounds exist =-=[13, 26]-=-, however, they are not related to our work. If graph-based, the arguments are often purely informationtheoretic, inthesensethatsomeinformation mustbe exchanged over some bottleneck link or node in a ... |

32 | Exploiting storage redundancy to speed up randomized shared memory simulations. Theoret
- Heide, Scheideler, et al.
- 1996
(Show Context)
Citation Context ...of view [5, 19]. Results related to ours have been discovered before for hashing problems. A number of publications presents algorithms with running times of O(log ∗ n) (or very close) in PRAM models =-=[4, 14, 27, 28]-=-. At the heart of these routines as well as our balls-into-bins solutions lies the idea to use an in each iteration exponentially growing share of the available resources to deal with the remaining ke... |

29 | Distributed selfish load balancing
- Berenbrink, Friedetzky, et al.
- 2006
(Show Context)
Citation Context ...to model tasks of varying resource consumption. The case of heterogeneous bins was examined as well [41]. In recent years, balls-into-bins has also been considered from a game theoretic point of view =-=[5, 19]-=-. Results related to ours have been discovered before for hashing problems. A number of publications presents algorithms with running times of O(log ∗ n) (or very close) in PRAM models [4, 14, 27, 28]... |

23 | Distributed MST for constant diameter graphs
- Lotker, Patt-Shamir, et al.
- 2001
(Show Context)
Citation Context ... the arguments are often purely informationtheoretic, inthesensethatsomeinformation mustbe exchanged over some bottleneck link or node in a carefully constructed network with diameter larger than two =-=[25, 35]-=-. In our setting, such information theoretic lower bounds will not work: Any two balls may exchange information along n edge-disjoint paths of length two, as the graph describing which edges could pot... |

23 |
A lower bound on probabilistic algorithms for distributive ring coloring
- Naor
- 1991
(Show Context)
Citation Context ...tially known to the respective bin only, but it may be forwarded during the course of an algorithm. 9 This general approach to argue about a simple topology has been popular when proving lower bounds =-=[9, 22, 34]-=-. communication may take the local topology of its current state into account. A different lower bound technique is by Kuhn et al. [21], where a specific locally symmetric, but globally asymmetric gra... |

18 |
W.: Fast Distributed Approximations in Planar Graphs
- Czygrinow, Hańćkowiak, et al.
- 2008
(Show Context)
Citation Context ...tially known to the respective bin only, but it may be forwarded during the course of an algorithm. 9 This general approach to argue about a simple topology has been popular when proving lower bounds =-=[9, 22, 34]-=-. communication may take the local topology of its current state into account. A different lower bound technique is by Kuhn et al. [21], where a specific locally symmetric, but globally asymmetric gra... |

18 | Parallel Balanced Allocations
- Stemann
- 1996
(Show Context)
Citation Context ...logn par. greedy [1] yes no 2 2.5 O O(n) loglogn par. greedy [1] yes no Θ ( ) loglogn Θ logloglogn ( ) loglogn O logloglogn ( ) loglogn O logloglogn ( ) nloglogn ( logloglogn ( ) ) logn 1/r collision =-=[38]-=- yes no 2 r +0.5 O O(n) loglogn A 2 b yes yes O(1) (exp.) log ∗ n+O(1) 2 O(n) log Ab(r) yes yes O(1) (exp.) r +O(1) (r) n log (r+1) +r +O(1) n O(n) Ac(l) yes yes O(l) (exp.) log ∗ n−log ∗ l+O(1) O(1) ... |

17 | Leveraging Linial’s Locality Limit
- Lenzen, Wattenhofer
- 2008
(Show Context)
Citation Context ...tially known to the respective bin only, but it may be forwarded during the course of an algorithm. 9 This general approach to argue about a simple topology has been popular when proving lower bounds =-=[9, 22, 34]-=-. communication may take the local topology of its current state into account. A different lower bound technique is by Kuhn et al. [21], where a specific locally symmetric, but globally asymmetric gra... |

16 |
The Log-Star Revolution
- Hagerup
- 1992
(Show Context)
Citation Context ... yes yes O(1) (exp.) 2 (2+o(1))n respectively. Implicitely, this approach already occured in previous work by Raman [37]. For a more detailed review of these papers, we refer the interested reader to =-=[16]-=-. Despite differences in the models, our algorithms and proofs exhibit quite a few structural similarities to the ones applicable to hashing in PRAM models. From our point of view, there are two main ... |

15 |
Fast and reliable parallel hashing
- Bast, Hagerup
- 1991
(Show Context)
Citation Context ...of view [5, 19]. Results related to ours have been discovered before for hashing problems. A number of publications presents algorithms with running times of O(log ∗ n) (or very close) in PRAM models =-=[4, 14, 27, 28]-=-. At the heart of these routines as well as our balls-into-bins solutions lies the idea to use an in each iteration exponentially growing share of the available resources to deal with the remaining ke... |

14 |
The power of collision: Randomized parallel algorithms for chaining and integer sorting
- Raman
- 1991
(Show Context)
Citation Context ...1+o(1))n adpt. greedy [8] yes yes 1+o(1) (exp.); at most d ≥ 2 O ( loglogn logd Aseq yes yes O(1) (exp.) 2 (2+o(1))n respectively. Implicitely, this approach already occured in previous work by Raman =-=[37]-=-. For a more detailed review of these papers, we refer the interested reader to [16]. Despite differences in the models, our algorithms and proofs exhibit quite a few structural similarities to the on... |

12 | Allocating Weighted Jobs in Parallel
- Berenbrink, Heide, et al.
- 1997
(Show Context)
Citation Context ...p norms instead of the maximum bin load (i.e., the L∞ norm) as quality measure, showing that the greedy strategy is p-competitive to an offline algorithm. Several works addressed weighted balls (e.g. =-=[6, 7, 20, 36, 39]-=-) in order to model tasks of varying resource consumption. The case of heterogeneous bins was examined as well [41]. In recent years, balls-into-bins has also been considered from a game theoretic poi... |

12 | Load balancing with memory
- Mitzenmacher, Prabhakar, et al.
- 2002
(Show Context)
Citation Context ...riations of the basic problem have been studied. Since resources often need to be assigned to dynamically arriving tasks, infinite processes have been considered (e.g. [3, 8, 29, 30, 31, 38, 40]). In =-=[32]-=- it is shown that, inthesequentialsetting, memorizinggoodchoicesfrom previousballs has similar impactas increasingthenumberof fresh random choices. Awerbuch et al. [2] studied arbitrary Lp norms inste... |

9 | Balanced Allocation on Graphs
- Kenthapadi, Panigrahy
- 2006
(Show Context)
Citation Context ... that dependency of random choices does not offer asymptotically better bounds. His upper bound holds also true if only two bins are chosen randomly, butfor each choice d/2 consecutivebins are queried=-=[18]-=-. Ta5 If balls cannot be allocated, they get an additional random choice. However, one could also give all balls this additional choice and let some of them ignore it, i.e., this kind of adaptivity ca... |

8 | Ballanced allocations with heterogeneous bins
- WIEDER
(Show Context)
Citation Context ... to an offline algorithm. Several works addressed weighted balls (e.g. [6, 7, 20, 36, 39]) in order to model tasks of varying resource consumption. The case of heterogeneous bins was examined as well =-=[41]-=-. In recent years, balls-into-bins has also been considered from a game theoretic point of view [5, 19]. Results related to ours have been discovered before for hashing problems. A number of publicati... |

6 | Balanced allocations: the weighted case
- TALWAR, WIEDER
(Show Context)
Citation Context ...p norms instead of the maximum bin load (i.e., the L∞ norm) as quality measure, showing that the greedy strategy is p-competitive to an offline algorithm. Several works addressed weighted balls (e.g. =-=[6, 7, 20, 36, 39]-=-) in order to model tasks of varying resource consumption. The case of heterogeneous bins was examined as well [41]. In recent years, balls-into-bins has also been considered from a game theoretic poi... |

5 |
On weighted balls-into-bins games
- BERENBRINK, FRIEDETZKY, et al.
(Show Context)
Citation Context |

4 | Load Balancing without Regret in the Bulletin Board Model
- Kleinberg, Piliouras, et al.
- 2009
(Show Context)
Citation Context ...to model tasks of varying resource consumption. The case of heterogeneous bins was examined as well [41]. In recent years, balls-into-bins has also been considered from a game theoretic point of view =-=[5, 19]-=-. Results related to ours have been discovered before for hashing problems. A number of publications presents algorithms with running times of O(log ∗ n) (or very close) in PRAM models [4, 14, 27, 28]... |

3 | The (1 + β)-Choice Process and Weighted Balls-into-Bins
- Peres, Talwar, et al.
- 1990
(Show Context)
Citation Context ... ) admit constant maximum bin load. It is noteworthy that for d ≥ 2 the imbalance between the most loaded bins and the average load is O(loglogn/logd) w.h.p. irrespective of m. Recently, Peres et al. =-=[36]-=- proved a similar result for the case where “d = 1 + β” bins are queried, i.e., balls choose with constant probability β ∈ (0,1) the least loaded of two bins, otherwise uniformly at random. In this se... |

2 |
Handbook of Randomized Computing, volume 1, chapter The Power of Two Random Choices: A Survey of the Techniques and Results
- Mitzenmacher, Richa, et al.
- 2001
(Show Context)
Citation Context ...aring work load among multiple processors, servers, or storage locations, but the problem also plays a vital role in e.g. low-congestion circuit routing, channel bandwidth assignment, or hashing, cf. =-=[33]-=-. A common archetype of all these tasks is the well-known balls-into-bins problem: Given n balls and n bins, how can one place the balls into the bins quickly while keeping the maximum bin load small?... |

1 |
Revisiting Randomized Parallel Load Balancing Algorithms
- Even, Medina
- 2009
(Show Context)
Citation Context ...∈ O(loglogn) achieves a load of O((logn/loglogn) 1/r ) w.h.p.; for r ∈ Θ(loglogn/logloglogn) this implies a constantly bounded bin load. Even and Medina also proposed a 2.5-round “adaptive” algorithm =-=[11]-=-. 5 Their synchronous algorithm uses a constant number of choices and exhibits a maximum bin load of Θ( √ logn/loglogn) w.h.p., i.e., exactly the same characteristics as parallel greedy with 2.5 round... |

1 |
Parallel Randomized Load Balancing: A Lower Bound for a More General Model
- Even, Medina
- 2010
(Show Context)
Citation Context ... r the maximum bin load is Ω((logn/loglogn) 1/r ) with constant probability. Recently, Even and Medina extended their bounds to a larger spectrum of algorithms by removing some artificial assumptions =-=[12]-=-. A matching algorithm was proposed by Stemann [38], which for d = 2 and r ∈ O(loglogn) achieves a load of O((logn/loglogn) 1/r ) w.h.p.; for r ∈ Θ(loglogn/logloglogn) this implies a constantly bounde... |

1 |
The Tree Model for Hashing: Lower and Upper Bounds
- Gil, Heide, et al.
- 1996
(Show Context)
Citation Context |