## Asymptotic Complexity from Experiments? -- A Case Study for Randomized Algorithms (2000)

Venue: | IN PROCEEDINGS OF THE 4TH WORKSHOP OF ALGORITHMS AND ENGINEERING (WAE'00 |

Citations: | 4 - 2 self |

### BibTeX

@INPROCEEDINGS{Sanders00asymptoticcomplexity,

author = {Peter Sanders and Rudolf Fleischer},

title = {Asymptotic Complexity from Experiments? -- A Case Study for Randomized Algorithms},

booktitle = {IN PROCEEDINGS OF THE 4TH WORKSHOP OF ALGORITHMS AND ENGINEERING (WAE'00},

year = {2000},

pages = {135--146},

publisher = {}

}

### OpenURL

### Abstract

In the analysis of algorithms we are usually interested in obtaining closed form expressions for their complexity, or at least asymptotic expressions in O(-)-notation. Unfortunately, there are fundamental reasons why we cannot obtain such expressions from experiments. This paper explains how we can at least come close to this goal using the scientific method. Besides the traditional role of experiments as a source of preliminary ideas for theoretical analysis, experiments can test falsifiable hypotheses obtained by incomplete theoretical analysis. Asymptotic behavior can also be deduced from stronger hypotheses which have been induced from experiments. As long as a complete mathematical analysis is impossible, well tested hypotheses may have to take their place. Several

### Citations

1313 | Introduction to parallel algorithms and architectures: array, trees, hypercubes - Leighton - 1992 |

616 | Mersenne twister: A 623-dimensionally equidistributed uniform pseudo-random number generator
- Matsumoto, Nishimura
- 1998
(Show Context)
Citation Context ...[16] below one percent of the average excess load. In order to minimize artifacts of the random number generator, we have used a generator with good reputation and very long period (2 19937 \Gamma 1) =-=[11]-=-. In addition, we have repeated some experiments with the Unix generator srand48 leading to almost identical results. Fig. 4 shows that a simple experiment at least predicts that l max \Gamma m=n cann... |

484 |
Introduction to Parallel Computing. Design and Analysis of Algorithms, The Benjamin/Cummings
- Kumar, Grama, et al.
- 1994
(Show Context)
Citation Context ...eading to friction between theory and practice. 4.2 "Heuristic" Deduction: Random Polling Let us consider the following simplified model for the startup phase of random polling dynamic load =-=balancing [9, 3, 17]-=- which is perhaps the best available algorithm for parallelizing tree shaped computations of unknown structure: There are n processing elements (PEs) numbered 0 through n \Gamma 1. At step t = 0, a ra... |

401 | Scheduling multithreaded computations by work stealing, in
- Blumofe, Leiserson
- 1994
(Show Context)
Citation Context ...eading to friction between theory and practice. 4.2 "Heuristic" Deduction: Random Polling Let us consider the following simplified model for the startup phase of random polling dynamic load =-=balancing [9, 3, 17]-=- which is perhaps the best available algorithm for parallelizing tree shaped computations of unknown structure: There are n processing elements (PEs) numbered 0 through n \Gamma 1. At step t = 0, a ra... |

251 | Balanced allocations
- Azar, Broder, et al.
- 1994
(Show Context)
Citation Context ...oach called balanced random allocation. Jobs are considered one after the other. Two random possible target PEs are chosen for each job and the job is allocated on the PE with lower load. Azar et al. =-=[1]-=- have shown that l max = O(m=n) + (1 + o(1)) log ln n whp for m = n. Interestingly, this bound shows that balanced random allocation is exponentially better than plain random allocation. However, for ... |

81 | How Asymmetry Helps Load Balancing
- Vöcking
- 2003
(Show Context)
Citation Context ...lanced allocation. Our experiments were done before the theoretical solution. Otherwise, we could have picked one of the other open problems in the area of balls into bins games. For example, Vocking =-=[21]-=- recently proved that an asymmetric placement rule for breaking ties can significantly reduce l max for m = n but nobody seems to know how to generalize this result for general m. 5 Discussion Assume ... |

73 | Needed: An empirical science of algorithms
- Hooker
- 1994
(Show Context)
Citation Context ...l Algorithmics) have been installed and established conferences (e.g., SODA, ESA) explicitly call for experimental work. Using the scientific method as a basis for algorithmics was proposed by Hooker =-=[6]-=-. McGeoch, Precup and Cohen [12] give heuristic algorithms for finding upper bounds on measured function values which are found to be reliable within a factor p n. They stress that finding more accura... |

72 |
Logik der Forschung
- Popper
- 1966
(Show Context)
Citation Context ...ults. Here, it is explained why with some care experiments can often play a stronger role. One way to make the meaning of "some care" more precise is to apply the terminology of the scientif=-=ic method [15]-=-. The scientific method views science as a cycle between theory and practice. Theory can inductively or (partially) deductively formulate falsifiable hypotheses which can be tested by experiments. The... |

58 | Vöcking: Balanced Allocations: The Heavily Loaded Case
- Berenbrink, Czumaj, et al.
(Show Context)
Citation Context ... exponentially better than plain random allocation. However, for large m their methods of analysis yield even weaker bounds than that for plain random allocation. Only very recently Berenbrink et al. =-=[2]-=- have shown (using quite nontrivial arguments) that l max = m=n + (1 + o(1)) log ln n : 3 We mean logical implication here, i.e., if the hypothesis is false nothing is said about the truth of the impl... |

50 | Fast concurrent access to parallel disks
- Sanders, Egner, et al.
(Show Context)
Citation Context ... 0.03 0 1 2 4 6 8 10 12 overhead 1-N/t W/D D/(2W) D=256 D=8 Fig. 1. Overhead (i.e., 1\Gammaefficiency) of EAGER. N = 10 6 \Delta D blocks were written. general technique for scheduling parallel disks =-=[18]-=-. We maintain one queue Q i for each disk. The queues share a buffer space of size W = O(D). We first put all the blocks into the queues and then write one block from each nonempty queue. If after tha... |

31 | Towards optimal locality in meshindexings
- Niedermeier, Reinhardt, et al.
- 1997
(Show Context)
Citation Context ... were first found experimentally and then proven analytically. Later it turned out that a class of experiments can be systematically converted into theoretical results valid for arbitrary curve sizes =-=[14]-=-. Experiments are more important for randomized algorithms. Often randomization can convert all instances into average case instances. For example, every sorting algorithm which is efficient on the av... |

31 |
A high-speed sorting procedure
- Shell
(Show Context)
Citation Context ...and A(tj) is the cumulative distribution function of the Student t-distribution withsdegrees of freedom. Within the computational precision of Maple, this probability is zero. 4.3 Shellsort Shellsort =-=[20]-=- is a classical sorting algorithm which is considered a good algorithm for almost sorted inputs in particular, if an in-place routine is desired or small to medium sized inputs are considered. Given a... |

30 |
Random duplicate assignment: An alternative to striping in video servers
- Korst
- 1997
(Show Context)
Citation Context ...nts with the Unix generator srand48 leading to almost identical results. Fig. 4 shows that a simple experiment at least predicts that l max \Gamma m=n cannot depend much on m. Other researchers (e.g. =-=[8]-=-) made some experiments but without trying to induce hypotheses on the asymptotic behavior of balanced allocation. Our experiments were done before the theoretical solution. Otherwise, we could have p... |

26 | Analysis of Shellsort and related algorithms, presented at the Fourth Annual European Symposium on Algorithms
- Sedgewick
- 1996
(Show Context)
Citation Context ...or inputs of the given size. On the other hand, closed form analysis of randomized algorithms can be very difficult. For example, the average case performance of Shellsort is open for a long time now =-=[19]-=-. Also refer to Sect. 4.3. Unbounded input size: Another problem with experiments is that we can only test a finite number of input sizes. For example, assume we find that some sorting algorithm needs... |

10 |
Lastverteilungsalgorithmen für parallele Tiefensuche. Number 463
- Sanders
- 1997
(Show Context)
Citation Context ...eading to friction between theory and practice. 4.2 "Heuristic" Deduction: Random Polling Let us consider the following simplified model for the startup phase of random polling dynamic load =-=balancing [9, 3, 17]-=- which is perhaps the best available algorithm for parallelizing tree shaped computations of unknown structure: There are n processing elements (PEs) numbered 0 through n \Gamma 1. At step t = 0, a ra... |

6 | Average-case complexity of Shellsort
- Jiang, Li, et al.
- 1999
(Show Context)
Citation Context ...of key comparisons performed by Shellsort for n inputs. It is unknown wether there is an offset sequence which yields a sorting algorithm with T (n) = O(n log n) or even one with T (n) = o(n log 2 n) =-=[19, 7]-=-. It is known that any algorithm with T (n) = O(n log n) must use \Theta(log n) offsets [7]. Previous experiments with many carefully constructed offset sequences led to the conjecture that no sequenc... |

5 |
Combinatorial Algorithms Test Sets (CATS): the ACM/EATCS platform for experimental research
- Goldberg, Moret
- 1999
(Show Context)
Citation Context ...put size because there are usually exponentially many of them. If we do application oriented research this problem may be mitigated by libraries of test instances which are considered "typical&qu=-=ot; (e.g. [5]-=-). Here we want to concentrate on experiments as a tool for theory however so that we are interested in cases where it is possible to bound the complexity for all inputs of size n in time polynomial i... |

4 | How to find big-oh in your data set (and how not to
- McGeoch, Precup, et al.
- 1997
(Show Context)
Citation Context ...ally. We only mention it as a problem in the current context of inducing asymptotic expressions from experiments. work in particular if we are interested in fine distinctions like logarithmic factors =-=[12]. Again, t-=-he scientific method helps to mitigate this problem. Often, we are able to handle a related or simplified version of the system analytically or we can make "heuristic" steps in a derivation ... |

4 |
Empirical study of the expected running time of Shellsort
- Weiss
- 1991
(Show Context)
Citation Context ...th T (n) = O(n log n) must use \Theta(log n) offsets [7]. Previous experiments with many carefully constructed offset sequences led to the conjecture that no sequence yields T (n) close to O(n log n) =-=[22]-=-. 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 4 16 64 256 1024 4096 2 14 2 16 2 18 2 20 2 22 #comparisons/log(n!) n Fig. 3. Competitive ratio of the average number of key comparisons of random offset Shells... |

4 |
Lexikon der Stochastik
- Müller
- 1991
(Show Context)
Citation Context ...red averages with standard deviation. We have tested the hypothesis by simulating the process 1000 times for n = 2 j and j 2 f1; : : : ; 16g. Fig. 2 shows the results. We have made the Student-t test =-=[13]-=- to determine the probability that the measured average number of steps only accidentally lies below the hypothesized bound. Within the computational precision of Maple, this probability is zero. On t... |

1 |
Lexikon der Stochastik. Akademie Verlag, 5th edition
- Muller
- 1991
(Show Context)
Citation Context ...lation which does not need the independence assumption. The probability that the measured values are only accidentally below the conjectured bound can be estimated using the Student-t test. Following =-=[13]-=- we get a probability 1 \Gamma A ` p 1000sT \Gamma (log n + log ln n + 1) oe j999 ' 0 5 10 15 20 32 1024 32768 n Measurement log n + log ln n + 1 log n Fig. 2. Number of random polling steps to get al... |