## On the Performance of Parallel Computers: Order Statistics and Amdahl's Law (1995)

Venue: | International Journal of Computers and Their Applications |

Citations: | 8 - 6 self |

### BibTeX

@ARTICLE{Zhang95onthe,

author = {Tao Zhang and Seonglim Kang and Lester Lipsky},

title = {On the Performance of Parallel Computers: Order Statistics and Amdahl's Law},

journal = {International Journal of Computers and Their Applications},

year = {1995},

volume = {3}

}

### OpenURL

### Abstract

this paper, we will give a short review of Amdahl's law, and order statistics, and provide some examples of what realistic speedups to expect. In particular, we will look at Uniform, Exponential, and Power-tail distributions in detail. Then we will examine the much more difficult problem where the number of tasks exceeds the number of processors. In this case the tasks must queue up for service. This is equivalent to Mean time to drain of a G/C queue with no new arrivals, and in Reliability Theory, to Mean time to failure, with hot and cold backup. We will also present some examples of this. We also give a description of how to treat jobs where the tasks come from different distributions, and show how longest task time can be computed if the distributions are all exponential (but with different parameters). In this case, if k exceeds C,

### Citations

1986 | On the self-similar nature of Ethernet traffic (extended version
- Lehand, Taqqu, et al.
- 1994
(Show Context)
Citation Context ...ributions, this behavior has been observed in numerous applications, including distribution of CPU times [LELA86], [LIPS86], size of files stored on disc [GARG92], and arrival of packets on ethernets =-=[LELA94]-=-. Details about these distributions can be found in [GREI95]. As an example, let the reliability function for task times be given by: R(x) = ` ff \Gamma 1 x + ff \Gamma 1 ' ff \Delta As long as ff ? 1... |

1746 |
An I..ntroduction to Probability Theory
- FELLER
- 1966
(Show Context)
Citation Context ..., one doesn't know what the distribution is, anyway) one can get a qualitative answer, where none is available otherwise. A surprising property concerning conditional proper8 ties described in Feller =-=[FELL71]-=- implies that if a job is broken up into k pieces in a random way, then the mean size of the largest piece depends on k in the same way as that for exponential service times! This in turn, implies tha... |

576 |
Validity of the Single Processor Approach to Achieving large Scale Computing Capabilities
- Amdahl
- 1967
(Show Context)
Citation Context ...we will call the former parallel tasks and the latter sequencial tasks. The length of a job or task is the time it takes to execute the job or task alone on a single processor. 2 Amdahl's Law In 1967 =-=[AMDA67]-=-, G. M. Amdahl made a simple and insightful argument about the limits of speedup from parallelization. He recognized that no matter what the job, some things must be done sequentially. i.e., some task... |

157 |
Probability and Statistics with Reliability
- Trivedi
(Show Context)
Citation Context ...o complete will be longer than the mean time for a task. How much longer requires some careful analysis. We give some more definitions so as to be consistent with standard texts on the subject (e.g., =-=[TRIV82]-=-). Let Y 1 ; Y 2 ; \Delta \Delta \Delta ; Y j ; \Delta \Delta \Delta ; Y k be the same set of random variables as the T j 's, except that they are in size place. That is, Y 1 is the smallest of the T ... |

123 |
Load-balancing heuristics and process behavior
- LELAND, OTT
- 1986
(Show Context)
Citation Context ...This in turn means that E(x ` ) = 1 8 `sff. As pathological as this may seem for practical distributions, this behavior has been observed in numerous applications, including distribution of CPU times =-=[LELA86]-=-, [LIPS86], size of files stored on disc [GARG92], and arrival of packets on ethernets [LELA94]. Details about these distributions can be found in [GREI95]. As an example, let the reliability function... |

93 |
Queueing Theory: A linear algebraic approach
- Lipsky
- 1992
(Show Context)
Citation Context ...ach (LAQT) to queueing theory, [LIPS83] for which programs have already been written [TEHR83], [ZHAN93]. We give a short discription of the LAQT approach to M/G/C queues here, and refer the reader to =-=[LIPS92]-=- for further details. 4.2.1 Matrix Representation of Distribution Functions Every distribution function can be approximated arbitrarily closely by some m-dimensional vector-matrix pair ! p ; B ? in th... |

32 |
The Importance of Power-Tail Distributions for Telecommunications Traffic Modeling,” Institute für Informatik, Technische Universität München
- Greiner, Jobmann, et al.
- 1995
(Show Context)
Citation Context ...cations, including distribution of CPU times [LELA86], [LIPS86], size of files stored on disc [GARG92], and arrival of packets on ethernets [LELA94]. Details about these distributions can be found in =-=[GREI95]-=-. As an example, let the reliability function for task times be given by: R(x) = ` ff \Gamma 1 x + ff \Gamma 1 ' ff \Delta As long as ff ? 1 this distribution has a mean of 1. From (7) we have E(Y k )... |

10 |
The Effect of Power-Tail Distributions on the Behavior of Time Sharing
- Garg, Lipsky, et al.
- 1992
(Show Context)
Citation Context ...thological as this may seem for practical distributions, this behavior has been observed in numerous applications, including distribution of CPU times [LELA86], [LIPS86], size of files stored on disc =-=[GARG92]-=-, and arrival of packets on ethernets [LELA94]. Details about these distributions can be found in [GREI95]. As an example, let the reliability function for task times be given by: R(x) = ` ff \Gamma 1... |

10 |
AutoCorrelation of Counting Processes Associated with Renewal Processes
- Lipsky, Fiorini
- 1995
(Show Context)
Citation Context ... E(Y k ) + I k ; where I 0 = 1 and for k ? 0 I k = Z 1 0 v k dv (1 \Gamma v) 1=ff = ffk ffk + ff \Gamma 1 I k\Gamma1 \Delta The last recursive relation came from integrating by parts. It can be shown =-=[LIPS95]-=- [and (1)] that E(Y k ) = Q(k) =) k 1=ff : Comparing with (10) we can see that the speedup for large k is much worse than that for exponential distributions. In fact, for ff ! 1 there is no speedup at... |

4 |
A Heuristic Fit of an Unusual Set of Data
- Lipsky
- 1986
(Show Context)
Citation Context ...rn means that E(x ` ) = 1 8 `sff. As pathological as this may seem for practical distributions, this behavior has been observed in numerous applications, including distribution of CPU times [LELA86], =-=[LIPS86]-=-, size of files stored on disc [GARG92], and arrival of packets on ethernets [LELA94]. Details about these distributions can be found in [GREI95]. As an example, let the reliability function for task ... |

3 |
Explicit Solutions of Generalized M/G/C//N Systems Including an Analysis of Their Transient Behaviour
- Tehranipour
- 1983
(Show Context)
Citation Context ...t with analytically. However, these problems have been treated in another guise, namely, the Linear Algebraic approach (LAQT) to queueing theory, [LIPS83] for which programs have already been written =-=[TEHR83]-=-, [ZHAN93]. We give a short discription of the LAQT approach to M/G/C queues here, and refer the reader to [LIPS92] for further details. 4.2.1 Matrix Representation of Distribution Functions Every dis... |

2 |
Explicit Solutions of M/G/C//N-type Queueing Loops with Generalizations
- Lipsky
- 1985
(Show Context)
Citation Context ...letion. The problem is virtually intractible when dealt with analytically. However, these problems have been treated in another guise, namely, the Linear Algebraic approach (LAQT) to queueing theory, =-=[LIPS83]-=- for which programs have already been written [TEHR83], [ZHAN93]. We give a short discription of the LAQT approach to M/G/C queues here, and refer the reader to [LIPS92] for further details. 4.2.1 Mat... |

1 |
Transient Properties of Parallel Queues
- Zhang
- 1993
(Show Context)
Citation Context ...lytically. However, these problems have been treated in another guise, namely, the Linear Algebraic approach (LAQT) to queueing theory, [LIPS83] for which programs have already been written [TEHR83], =-=[ZHAN93]-=-. We give a short discription of the LAQT approach to M/G/C queues here, and refer the reader to [LIPS92] for further details. 4.2.1 Matrix Representation of Distribution Functions Every distribution ... |