## Fitting Mixtures Of Exponentials To Long-Tail Distributions To Analyze Network Performance Models (1997)

### Cached

### Download Links

- [www.research.att.com]
- [www.net.uni-sb.de]
- [wnlab.isti.cnr.it]
- DBLP

### Other Repositories/Bibliography

Citations: | 142 - 13 self |

### BibTeX

@INPROCEEDINGS{Feldmann97fittingmixtures,

author = {Anja Feldmann and Ward Whitt},

title = {Fitting Mixtures Of Exponentials To Long-Tail Distributions To Analyze Network Performance Models},

booktitle = {},

year = {1997},

pages = {245--279}

}

### OpenURL

### Abstract

Traffic measurements from communication networks have shown that many quantities characterizing network performance have long-tail probability distributions, i.e., with tails that decay more slowly than exponentially. File lengths, call holding times, scene lengths in MPEG video streams, and intervals between connection requests in Internet traffic all have been found to have long-tail distributions, being well described by distributions such as the Pareto and Weibull. It is known that long-tail distributions can have a dramatic effect upon performance, e.g., long-tail service-time distributions cause long-tail waiting-time distributions in queues, but it is often difficult to describe this effect in detail, because performance models with component long-tail distributions tend to be difficult to analyze. We address this problem by developing an algorithm for approximating a long-tail distribution by a hyperexponential distribution (a finite mixture of exponentials). We first prove tha...

### Citations

3016 |
Probability and Measure
- Billingsley
- 1986
(Show Context)
Citation Context ..."close" and upon regularity conditions. For probability distributions on the real line (or, more generally, on a metric space) it is customary to use the notion of weak convergence, as in Bi=-=llingsley [11]-=-. In that framework, we say that a sequence of probability measures fP n : ns1g converges to a probability measure P , and write P n ) P , if Z fdP n ! Z fdP as n !1 (2.1) for all bounded continuous r... |

1879 |
An Introduction to Probability Theory and its Applications
- Feller
- 1967
(Show Context)
Citation Context ...ility density function (pdf) f is said to be completely monotone if all derivatives of f exist and (\Gamma1) n f (n) (t)s0 for all t ? 0 and ns1 ; (3.1) see p. 66 of Keilson [33] and p. 439 of Feller =-=[24]-=-. the link between completely monotone pdf's and mixtures of exponential pdf's is provided by Bernstein's [10] theorem. (see [24]). 9 Algorithm fit Moment fit Waiting-time Simulated Weibull Calc. Hype... |

1765 | On the Self-Similar Nature of Ethernet Traffic (Extended Version - Leland, Taqqu, et al. - 1994 |

1409 | Wide area traffic: The failure of Poisson modeling
- Paxson, Floyd
- 1995
(Show Context)
Citation Context ...]), and Willinger, et al. [59] have proved that such long-tailed on and off times for individual sources can explain the self-similarity in the aggregate traffic. Paxson [45, 46] and Paxson and Floyd =-=[47]-=- find that long-tail distributions yield statistically better models for the tail behavior of durations, number of bytes, and burst bytes of ftp connections on the Internet. Feldmann [22, 23] has show... |

1129 | Self-similarity in World Wide Web traffic: evidence and possible causes
- Crovella, Bestavros
- 1997
(Show Context)
Citation Context ...tion requests in Internet traffic have long-tail distributions. C'aceres et. al [14] present further evidence of long-tail distributions in Internet traffic. Recent analysis by Crovella and Bestavros =-=[18]-=- of the durations of world wide web transfers have led to scrutinizing the file length distribution on file servers. Both distributions have been found to be long-tailed. Mogul's [40] investigation of... |

527 |
Applied probability and queues
- Asmussen
- 2003
(Show Context)
Citation Context ...l interarrival-time and service-time distributions. Fortunately, such robustness, stability or continuity properties have been established for performance models, e.g., see Section VIII.5 of Asmussen =-=[5]-=-, Section 21 of Borovkov [13], Kalashnikov and Rachev [32] and Whitt [52], [53]. Even though robustness results have been established, care is needed because the robustness results do not hold uncondi... |

273 |
Structured stochastic matrices of M/G/1 type and their applications. New York etc
- Neuts
- 1989
(Show Context)
Citation Context ...ic model is the Markovian arrival process (MAP) or its extension, the batch Markovian arrival process (BMAP), also known as the virtual Markovian point process, see Lucantoni [36], Chapter 5 of Neuts =-=[43]-=-, and Andersen et al. [4]. The potential power of a MAP is dramatically demonstrated by a theoretical result due to Asmussen and Koole [7]. They proved that any stationary point process can be approxi... |

181 | Large deviations and overflow probabilities for the general single-server queue, with applications
- Duffield, OIConnell
- 1993
(Show Context)
Citation Context ...g-time distribution in a single-server queue with unlimited waiting space inherits the long-tail property of a service-time distribution with a power tail. For more recent work in this direction, see =-=[1, 6, 16, 19, 20, 25, 29]-=-. However, the impact of a long-tail distribution depends on the context and requires careful analysis. For example, in the single-server queue, large delays are caused by large service times and shor... |

173 | Packet Trains: Measurements and a New Model for Computer Network Traffic
- Jain, Routhier
- 1985
(Show Context)
Citation Context ...ibull(:3, 9:261) distribution using c k = 0:0001 and c 1 = 5. 32 8. Making Markov-Modulated On-Off Sources A commonly considered model for sources in communication networks is the on-off model (e.g., =-=[4, 15, 16, 28, 37, 49, 59]-=-). In the basic on-off source model, the on and off periods come from independent sequences of i.i.d. random variables, with the on periods having cdf F 1 and the off periods having another common cdf... |

149 | The Fourier-series method for inverting transforms of probability distributions. Queueing Systems Theory Appl
- Abate, Whitt
- 1992
(Show Context)
Citation Context ...mulations are based on a time period of 5:3 \Theta 10 6 , which corresponds to about 4 \Theta 10 6 arrivals. The numerical results for the M=H 6 =1 model are obtained by numerical transform inversion =-=[2]-=-. Figure 4 (a)--(d) indicate that the hyperexponential distribution with 6 exponentials is a good approximation to the Weibull distribution with c = 0:6. Accordingly, it is no surprise that Figure 4 (... |

129 |
Stochastic Processes in Queueing Theory
- Borovkov
- 1976
(Show Context)
Citation Context ...ice-time distributions. Fortunately, such robustness, stability or continuity properties have been established for performance models, e.g., see Section VIII.5 of Asmussen [5], Section 21 of Borovkov =-=[13]-=-, Kalashnikov and Rachev [32] and Whitt [52], [53]. Even though robustness results have been established, care is needed because the robustness results do not hold unconditionally. The robustness depe... |

121 |
Self-similarity through high variability: statistical analysis of Ethernet LAN traffic at source level
- Willingsler, Taqqu, et al.
(Show Context)
Citation Context ...s of source on and off times (high and low activity times) of individual network sources within the Bellcore dataset have indicated long-tail distributions (Leland et al. [35]), and Willinger, et al. =-=[59]-=- have proved that such long-tailed on and off times for individual sources can explain the self-similarity in the aggregate traffic. Paxson [45, 46] and Paxson and Floyd [47] find that long-tail distr... |

117 |
Empirically Derived Analytic Models of Wide-Area TCP
- Paxson
- 1994
(Show Context)
Citation Context ...stributions (Leland et al. [35]), and Willinger, et al. [59] have proved that such long-tailed on and off times for individual sources can explain the self-similarity in the aggregate traffic. Paxson =-=[45, 46]-=- and Paxson and Floyd [47] find that long-tail distributions yield statistically better models for the tail behavior of durations, number of bytes, and burst bytes of ftp connections on the Internet. ... |

109 | SelfSimilarity in high-speed packet traffic: Analysis and modeling of Ethernet traffic measurements - Leland, Taqqu, et al. - 1995 |

100 | Characteristics of wide-area TCP/IP conversations
- Caceres, Danzig, et al.
(Show Context)
Citation Context ...bytes, and burst bytes of ftp connections on the Internet. Feldmann [22, 23] has shown that the intervals between connection requests in Internet traffic have long-tail distributions. C'aceres et. al =-=[14]-=- present further evidence of long-tail distributions in Internet traffic. Recent analysis by Crovella and Bestavros [18] of the durations of world wide web transfers have led to scrutinizing the file ... |

97 | Network behavior of a busy Web server and its clients
- Mogul
- 1995
(Show Context)
Citation Context ...la and Bestavros [18] of the durations of world wide web transfers have led to scrutinizing the file length distribution on file servers. Both distributions have been found to be long-tailed. Mogul's =-=[40]-=- investigation of a very busy world-wide-web server indicates that interarrival times of accesses have long tails. Jelenkovi'c et al. [30] find that the lengths of scenes in MPEG video streams have a ... |

84 |
Statistical Theory of Reliability and Life
- Barlow, Proschan
- 1975
(Show Context)
Citation Context ... dt ln(F c (t)j t=c i = f(c i ) F c (c i ) = r(c i ) ; (5.2) which is the hazard rate function (or failure rate function) associated with the ccdf F c evaluated at c i ; e.g., see Barlow and Proschan =-=[9]-=-. Indeed, we could consider (4.10) replaced with (5.2), but (4.10) seems more robust. The idea in the procedure of Section 4 is to havesi be significantly less thansi+1 for all i. In order to havesi b... |

79 |
Markov chain models - rarity and exponentiality
- Keilson
- 1979
(Show Context)
Citation Context ...y monotone pdf's. A probability density function (pdf) f is said to be completely monotone if all derivatives of f exist and (\Gamma1) n f (n) (t)s0 for all t ? 0 and ns1 ; (3.1) see p. 66 of Keilson =-=[33]-=- and p. 439 of Feller [24]. the link between completely monotone pdf's and mixtures of exponential pdf's is provided by Bernstein's [10] theorem. (see [24]). 9 Algorithm fit Moment fit Waiting-time Si... |

73 |
Approximating a Point Process by a Renewal Process, I: Two Basic Methods
- Whitt
- 1982
(Show Context)
Citation Context ...mixture of two exponentials, which has three parameters) to the Weibull distribution by matching the first three moments. A three-moment matching algorithm for the H 2 fit is given on p. 136 of Whitt =-=[54]-=-. the resulting H 2 parameters are p 1 = 0:00501,s1 = 0:019, ands2 = 1:355. The approximating H 2 density and ccdf are compared to their Weibull counterparts in Figure 1 (a), (b). It is obvious that t... |

72 | Squeezing the most out of ATM
- Choudhury, Lucantoni, et al.
- 1996
(Show Context)
Citation Context ... no difficulties. It is challenging to analyze models with elaborate MAPs and BMAPs constructed to capture complex traffic behavior, but new effective computational schemes are being developed, e.g., =-=[15]-=-, [37]. This is also not to say that new models should not be sought and examined. However, the main theme of this paper is that there is more that we can do with the tools at hand than might be expec... |

57 | The effect of multiple time scales and subexponentiality in MPEG video streams on queueing behavior
- Jelenkovic, Lazar, et al.
- 1997
(Show Context)
Citation Context ...Both distributions have been found to be long-tailed. Mogul's [40] investigation of a very busy world-wide-web server indicates that interarrival times of accesses have long tails. Jelenkovi'c et al. =-=[30]-=- find that the lengths of scenes in MPEG video streams have a long-tail distribution. Izquierdo and Reeves [27] show that the number of cells in VBR encoded video sequences has a long-tail distributio... |

55 | Waiting-time tail probabilities in queues with long-tail service-time distributions. Queueing Systems 16(3/4
- Abate, Choudhury, et al.
- 1994
(Show Context)
Citation Context ...g-time distribution in a single-server queue with unlimited waiting space inherits the long-tail property of a service-time distribution with a power tail. For more recent work in this direction, see =-=[1, 6, 16, 19, 20, 25, 29]-=-. However, the impact of a long-tail distribution depends on the context and requires careful analysis. For example, in the single-server queue, large delays are caused by large service times and shor... |

53 |
Some results on regular variation for distributions in queueing and fluctuation
- Cohen
- 1973
(Show Context)
Citation Context ...ility distributions associated with network traffic have long tails. Moreover, it is known that long-tail distributions can have a dramatic impact upon network performance. For example, in 1973 Cohen =-=[17]-=- showed that the steady-state waiting-time distribution in a single-server queue with unlimited waiting space inherits the long-tail property of a service-time distribution with a power tail. For more... |

46 |
Approximations for the GI/G/m queue
- Whitt
- 1993
(Show Context)
Citation Context ...vice-time distribution directly. This reflects the extensive experience showing that approximations based on two moments of the interarrival-time and service-time distributions can be quite effective =-=[57]-=-. However, even though the M=H 2 =1 approximation might be good enough for some engineering applications, the M=H 20 =1 approximation in (c) and (d) is far better. This is perhaps more evident from Ta... |

45 | Subexponential asymptotics of a Markovmodulated random walk with queueing applications
- Jelenković, Lazar
- 1998
(Show Context)
Citation Context ...g-time distribution in a single-server queue with unlimited waiting space inherits the long-tail property of a service-time distribution with a power tail. For more recent work in this direction, see =-=[1, 6, 16, 19, 20, 25, 29]-=-. However, the impact of a long-tail distribution depends on the context and requires careful analysis. For example, in the single-server queue, large delays are caused by large service times and shor... |

40 |
Marked point processes as limits of Markovian arrival streams
- Asmussen, Koole
- 1993
(Show Context)
Citation Context ...kovian point process, see Lucantoni [36], Chapter 5 of Neuts [43], and Andersen et al. [4]. The potential power of a MAP is dramatically demonstrated by a theoretical result due to Asmussen and Koole =-=[7]. They pro-=-ved that any stationary point process can be approximated arbitrarily closely by a MAP. (The meaning of "close" is defined in Section 2.) This is not to say that there are no difficulties. I... |

40 |
Modeling Call Holding Time Distributions for CCS Network Design and Performance Analysis
- Bolotin
- 1994
(Show Context)
Citation Context ...ves [27] show that the number of cells in VBR encoded video sequences has a long-tail distribution. Even telephone call holding-time distributions have been found to be long-tailed; e.g., see Bolotin =-=[12]-=- and Duffy et al. [21]. The accumulated evidence is clear: many important probability distributions associated with network traffic have long tails. Moreover, it is known that long-tail distributions ... |

39 | Economies of scale in queues with sources having power-law large deviation scalings
- Duffield
- 1994
(Show Context)
Citation Context |

36 |
Traffic models for ISDN data users: Office automation application
- Meier-Hellstern, Wirth, et al.
- 1991
(Show Context)
Citation Context ...recent years; see Pawlita [44]. Marshall and Morgan [38] note that the empirical distributions of local-area network traffic have longer tails than an exponential distribution. Meier-Hellstern et al. =-=[39]-=- observed high variability in their interarrival times of packets that seems best described with long-tail distributions. The analysis of a large dataset of local area Internet IP traffic collected at... |

29 |
Continuity of generalized semi-markov processes
- Whitt
- 1980
(Show Context)
Citation Context ...ss, stability or continuity properties have been established for performance models, e.g., see Section VIII.5 of Asmussen [5], Section 21 of Borovkov [13], Kalashnikov and Rachev [32] and Whitt [52], =-=[53]. Even tho-=-ugh robustness results have been established, care is needed because the robustness results do not hold unconditionally. The robustness depends upon what we mean by "close" and upon regulari... |

28 | Veciana, “On the relevance of time scales in performance oriented traffic characterizations
- Montgomery, De
- 1996
(Show Context)
Citation Context ... in the statistical analysis of traffic measurements (e.g., Leland et al. [35]). There is also strong evidence of important phenomena at several different time scales (e.g., Montgomery and de Veciana =-=[41]-=-). The complexity revealed by these traffic measurements have led some to suggest that this traffic cannot be analyzed by available traffic models. However, we contend that available traffic models ca... |

26 | Long-tail buffer-content distributions in broadband networks, Performance Evaluation 30
- Choudhury, Whitt
- 1997
(Show Context)
Citation Context |

25 |
Fitting phase type distributions via the EM algorithm
- Asmussen, Nerman, et al.
- 1996
(Show Context)
Citation Context ...ematical program. A natural alternative is the expectation-maximization (EM) algorithm, which is an iterative procedure that minimizes the Kullback-Leibler "distance"; see Asmussen, Nerman a=-=nd Olsson [8]-=-, Turin [51] and references therein. A difficulty with the EM algorithm is that the iteration can be slow when there are many parameters. The EM algorithm can be enhanced significantly if a good start... |

24 |
Large claims approximations for risk processes in a Markovian environment, Stochastic Process
- Asmussen, Henriksen, et al.
- 1994
(Show Context)
Citation Context |

24 |
Performance Analysis of Digital Transmission Systems
- Turin
- 1990
(Show Context)
Citation Context ...meters. After deriving this recursive fitting procedure, we learned that the general recursive estimation procedure actually has a long history, being know as Prony's [48] method; see p. 114 of Turin =-=[50]-=-. In that context, we contribute by showing when the recursive fitting procedure should be effective (Sections 3 and 5 here) and by applying it to approximate long-tail distributions. Here is the proc... |

21 | An operational calculus for probability distributions via Laplace transforms
- Abate, Whitt
- 1996
(Show Context)
Citation Context ...tribution is convenient because its Laplace transform is available. (See Section 6.3 below.) Other methods for constructing long-tail distributions with convenient Laplace transforms are described in =-=[3]-=-. In order to approximate a completely monotone cdf F having spectral cdf G by a hyperexponential distribution (a finite mixture of exponentials), it suffices to approximate the spectral cdf G by a sp... |

21 |
The BMAP/G/1 queue: A tutorial,” in Models and Techniques for Performance Evaluation of
- Lucantoni
- 1993
(Show Context)
Citation Context ... of a more powerful traffic model is the Markovian arrival process (MAP) or its extension, the batch Markovian arrival process (BMAP), also known as the virtual Markovian point process, see Lucantoni =-=[36]-=-, Chapter 5 of Neuts [43], and Andersen et al. [4]. The potential power of a MAP is dramatically demonstrated by a theoretical result due to Asmussen and Koole [7]. They proved that any stationary poi... |

16 |
Modelling characteristics of tcp connections
- Feldmann
- 1996
(Show Context)
Citation Context ...on and Floyd [47] find that long-tail distributions yield statistically better models for the tail behavior of durations, number of bytes, and burst bytes of ftp connections on the Internet. Feldmann =-=[22, 23]-=- has shown that the intervals between connection requests in Internet traffic have long-tail distributions. C'aceres et. al [14] present further evidence of long-tail distributions in Internet traffic... |

16 |
The transient BMAP/G/1 queue. Stochastic Models
- Lucantoni, Choudhury, et al.
- 1994
(Show Context)
Citation Context ...fficulties. It is challenging to analyze models with elaborate MAPs and BMAPs constructed to capture complex traffic behavior, but new effective computational schemes are being developed, e.g., [15], =-=[37]-=-. This is also not to say that new models should not be sought and examined. However, the main theme of this paper is that there is more that we can do with the tools at hand than might be expected. I... |

10 |
On-line call admission for high-speed networks
- Feldmann
- 1995
(Show Context)
Citation Context ...on and Floyd [47] find that long-tail distributions yield statistically better models for the tail behavior of durations, number of bytes, and burst bytes of ftp connections on the Internet. Feldmann =-=[22, 23]-=- has shown that the intervals between connection requests in Internet traffic have long-tail distributions. C'aceres et. al [14] present further evidence of long-tail distributions in Internet traffic... |

9 | Statistical characterization of MPEG VBR video at the SLICE layer
- Izquierdo, Reeves
- 1995
(Show Context)
Citation Context ...server indicates that interarrival times of accesses have long tails. Jelenkovi'c et al. [30] find that the lengths of scenes in MPEG video streams have a long-tail distribution. Izquierdo and Reeves =-=[27]-=- show that the number of cells in VBR encoded video sequences has a long-tail distribution. Even telephone call holding-time distributions have been found to be long-tailed; e.g., see Bolotin [12] and... |

9 |
Two decades of data traffic measurements: A survey of published results,experiences and applicability
- Pawlita
- 1988
(Show Context)
Citation Context ...f c ! 1, and we will only consider that case. There has been a long history traffic measurements, but the identification of long-tail distributions has been a major theme in recent years; see Pawlita =-=[44]-=-. Marshall and Morgan [38] note that the empirical distributions of local-area network traffic have longer tails than an exponential distribution. Meier-Hellstern et al. [39] observed high variability... |

9 |
Fitting probabilistic automata via the EM algorithm
- Turin
- 1996
(Show Context)
Citation Context ...ogram. A natural alternative is the expectation-maximization (EM) algorithm, which is an iterative procedure that minimizes the Kullback-Leibler "distance"; see Asmussen, Nerman and Olsson [=-=8], Turin [51]-=- and references therein. A difficulty with the EM algorithm is that the iteration can be slow when there are many parameters. The EM algorithm can be enhanced significantly if a good starting point ca... |

8 |
Statistics of Mixed Data Traffic on a Local Area Network
- Marshall, Morgan
- 1985
(Show Context)
Citation Context ...consider that case. There has been a long history traffic measurements, but the identification of long-tail distributions has been a major theme in recent years; see Pawlita [44]. Marshall and Morgan =-=[38]-=- note that the empirical distributions of local-area network traffic have longer tails than an exponential distribution. Meier-Hellstern et al. [39] observed high variability in their interarrival tim... |

8 |
On approximations for queues, III: mixtures of exponential distributions
- Whitt
- 1984
(Show Context)
Citation Context ...tail distribution depends on the context and requires careful analysis. For example, in the single-server queue, large delays are caused by large service times and short interarrival times, e.g., see =-=[55, 1, 16]-=-. In some distributions, long tails imply that small values are more likely too, but exceptionally long interarrival times by themselves typically do not cause large delays. Not only are long-tail dis... |

7 |
Modelling and performance study of packet–traffic with self–similar characteristics over several timescales with Markovian Arrival Processes (MAP
- Andersen, Jensen, et al.
- 1995
(Show Context)
Citation Context ...arrival process (MAP) or its extension, the batch Markovian arrival process (BMAP), also known as the virtual Markovian point process, see Lucantoni [36], Chapter 5 of Neuts [43], and Andersen et al. =-=[4]-=-. The potential power of a MAP is dramatically demonstrated by a theoretical result due to Asmussen and Koole [7]. They proved that any stationary point process can be approximated arbitrarily closely... |

7 |
Le Boudec, “Can self-similar traffic be modeled by Markovian processes
- Robert, Y
(Show Context)
Citation Context ...ibull(:3, 9:261) distribution using c k = 0:0001 and c 1 = 5. 32 8. Making Markov-Modulated On-Off Sources A commonly considered model for sources in communication networks is the on-off model (e.g., =-=[4, 15, 16, 28, 37, 49, 59]-=-). In the basic on-off source model, the on and off periods come from independent sequences of i.i.d. random variables, with the on periods having cdf F 1 and the off periods having another common cdf... |

6 |
Waiting times when service times are stable laws: tamed and wild
- Gaver, Jacobs
- 1998
(Show Context)
Citation Context |

6 |
Planning queueing simulations, Management Sci
- Whitt
- 1989
(Show Context)
Citation Context ...ay be meaningless to compare the simulation results to the calculated results if the problem is sensitive to the mean, which is the case for service-time distributions in queueing models. As shown in =-=[56]-=-, obtaining good simulation estimates of queueing characteristics becomes increasingly difficult as service-time variability increases. Moreover, the approach to steady state gets very slow, so that i... |

3 |
The Pareto distribution as a queue service distribution
- Harris
- 1968
(Show Context)
Citation Context ...ical induction) , it is easy to see that the pdf's of the Pareto distribution in (1.5) and the Weibull distribution 10 with a ! 1 in (1.6) are completely monotone. For the Pareto distribution, Harris =-=[26]-=- directly showed that the spectral cdf is gamma. (This is an easy calculation; see [26] or [31].) The gamma pdf with shape parameter less than 1 is also completely monotone. The Pareto mixture of expo... |