## Multicast-Based Inference of Network-Internal Characteristics: Accuracy of Packet Loss Estimation (1998)

### Cached

### Download Links

- [www.kiskeya.net]
- [kiskeya.org]
- [www.kiskeya.net]
- [kiskeya.org]
- [eeweb.poly.edu]
- [www.cs.utexas.edu]
- [www.ce.uniroma2.it]
- [www.research.att.com]
- [www.research.att.com]
- [gaia.cs.umass.edu]
- [gaia.cs.umass.edu]
- [gaia.cs.umass.edu]
- [www.research.att.com]
- [www.research.att.com]
- [www.research.att.com]
- DBLP

### Other Repositories/Bibliography

Venue: | IEEE Transactions on Information Theory |

Citations: | 251 - 31 self |

### BibTeX

@ARTICLE{Caceres98multicast-basedinference,

author = {R. Caceres and N.G. Duffield and J. Horowitz and D. Towsley and T. Bu},

title = {Multicast-Based Inference of Network-Internal Characteristics: Accuracy of Packet Loss Estimation},

journal = {IEEE Transactions on Information Theory},

year = {1998},

volume = {45},

pages = {2462--2480}

}

### Years of Citing Articles

### OpenURL

### Abstract

We explore the use of end-to-end multicast traffic as measurement probes to infer network-internal characteristics. We have developed in an earlier paper [2] a Maximum Likelihood Estimator for packet loss rates on individual links based on losses observed by multicast receivers. This technique exploits the inherent correlation between such observations to infer the performance of paths between branch points in the multicast tree spanning the probe source and its receivers. We evaluate through analysis and simulation the accuracy of our estimator under a variety of network conditions. In particular, we report on the error between inferred loss rates and actual loss rates as we vary the network topology, propagation delay, packet drop policy, background traffic mix, and probe traffic type. In all but one case, estimated losses and probe losses agree to within 2 percent on average. We feel this accuracy is enough to reliably identify congested links in a wide-area internetwork. Keywords---Internet performance, end-to-end measurements, Maximum Likelihood Estimator, tomography I.

### Citations

2270 |
Congestion avoidance and control
- Jacobson
(Show Context)
Citation Context ...gh queue overflows due to TCP interference do not obey our temporal independence assumption. TCP is a bursty packet source, particularly in the region of exponential window growth during a slow start =-=[9]-=-. In our simulations, multicast probes are often lost in groups as they compete for queue space with TCP bursts. This phenomenon is readily apparent when watching animations of our simulations with th... |

2185 | Random early detection gateways for congestion avoidance
- FLOYD, V
- 1993
(Show Context)
Citation Context ...sting spatial correlations are unlikely in a real network such as the Internet because of its traffic and link diversity. Furthermore, we believe that the introduction of Random Early Detection (RED) =-=[6]-=- policies in Internet routers will help break such correlations. In any case, our analysis shows that the introduction of correlations introduces inference errors in a continuous manner: if the correl... |

1907 | RTP: A Transport Protocol for Real-Time Applications
- Schulzrinne, Casner, et al.
- 1996
(Show Context)
Citation Context ...st measurement has been recently proposed in [2]. This approach, which does not require specialized end hosts, relies on the use the Real-Time Transport Protocol (RTP) and its control protocol (RTCP) =-=[31]-=-. RTP is used to carry multicast audio and video over the Internet. RTCP is used by the receivers to periodically multicast reports to the group for transmission control.TheideaistoregardanyongoingRTP... |

1374 |
Functional analysis
- Rudin
- 1991
(Show Context)
Citation Context ...e e.g. [14]). Then, the observed probabilities " converge to the true values ! with probability ! 1 as the number of observations . A simple argument involving the Inverse Function Theorem (e.g., see =-=[24]-=-) shows : that when the observations then the observed frequencies " Chapter 6 of [3]). / 1 is continuous on : , and hence " with probability 1. As a specific example, form a Markov chain (and also un... |

833 | Transmission control protocol - Postel - 1988 |

718 | End-to-end internet packet dynamics
- Paxson
- 1997
(Show Context)
Citation Context ...f end-to-end performance is not completely understood. Regarding the second approach, there has been much recent experimental work to understand the phenomenology of end-to-end performance (e.g., see =-=[1, 2, 13, 18, 20, 21]-=-). A number of ongoing measurement infrastructure projects (Felix [5], IPMA [7], NIMI [12] and Surveyor [26]) aim to collect and analyze endto-end measurements across a mesh of paths between a number ... |

657 |
Large Deviation Techniques and Applications
- Dembo, Zeitouni
- 1993
(Show Context)
Citation Context ...obability ! 1 as the number of observations . A simple argument involving the Inverse Function Theorem (e.g., see [24]) shows : that when the observations then the observed frequencies " Chapter 6 of =-=[3]-=-). / 1 is continuous on : , and hence " with probability 1. As a specific example, form a Markov chain (and also under more general mixing conditions) will converge exponentially fast to their mean va... |

575 | End-to-end routing behavior in the internet
- Paxson
- 1996
(Show Context)
Citation Context ...ion is made regarding the cooperation of network elements on the path. There has been much recent experimental work to understand the phenomenology of end-to-end performance (e.g., see [8], [19], and =-=[25]-=-–[27]). Several research efforts are developing measurement infrastructures (Felix [12], IPMA [14], NIMI [18], and Surveyor [32]) with the aim of collecting and analyzing end-to-end measurements acros... |

545 |
Markov Chains and Stochastic Stability
- Meyn, Tweedie
- 1993
(Show Context)
Citation Context ...ce of sufficiently short-range temporal correlations, provided that the underlying processes are stationary and ergodic. (This happens, for example, when recurrence conditions are satisfied; see e.g. =-=[14]-=-). Then, the observed probabilities " converge to the true values ! with probability ! 1 as the number of observations . A simple argument involving the Inverse Function Theorem (e.g., see [24]) shows... |

536 |
Theory of statistics
- Schervish
- 1995
(Show Context)
Citation Context ... the MLEs. The Fisher Information Matrix at based on is a mapping from to the = -dimensional real matrices defined by . It is straightforward to verify that satisfies conditions (see Section 2.3.1 of =-=[25]-=-) under which is equal to the following more convenient expression which we will use in the sequel: Theorem 4 When is non-singular, then as , under , to a = ' -dimensional Gaussian random variable wit... |

483 | Solving Least Squares Problems - Lawson, Hanson - 1974 |

442 | Wide-area internet traffic patterns and characteristics
- Thompson, J, et al.
- 1997
(Show Context)
Citation Context ...ations, losses were due to queue overflows as multicast probes competed with other traffic generated by infinite TCP sources. We used TCP because it is the dominant transport protocol in the Internet =-=[27]-=-. The following two subsections describe our results from these two simulation efforts. 5.1 Model Simulations Topology. For the model simulations, we used ad hoc software written in C++. We simulated ... |

438 | Network time protocol (version 3): Speci cation, implementation and analysis
- Mills
- 1992
(Show Context)
Citation Context ...chronization to within tenths of microseconds; it is currently used or planned in several of the measurement infrastructures mentioned earlier. More widely deployed is the Network Time Protocol (NTP) =-=[20]-=-. However, this provides accuracy only on the order of milliseconds at best, a resolution at least as coarse as the queueing delays in practice. Alternative approaches that can supplement delay measur... |

363 | Measurements and Analysis of End-to-End Internet Dynamics
- Paxson
- 1997
(Show Context)
Citation Context ... exponentially distributed spacings will see time 18averages; this is the PASTA property (Poisson Arrivals See Time Averages; see e.g. [30]). This approach has been proposed for network measurements =-=[22]-=- and is under consideration in the IP Performance Metrics working group of the IETF [8]. In the context of the above discussion, lengthening the interprobe time is to be understood as increasing the m... |

273 | Measuring bottleneck link speed in packet-switched networks
- Carter, Crovella
- 1996
(Show Context)
Citation Context ...here no assumption is made regarding the cooperation of network elements on the path. There has been much recent experimental work to understand the phenomenology of end-to-end performance (e.g., see =-=[8]-=-, [19], and [25]–[27]). Several research efforts are developing measurement infrastructures (Felix [12], IPMA [14], NIMI [18], and Surveyor [32]) with the aim of collecting and analyzing end-to-end me... |

250 | Using pathchar to estimate Internet link characteristics
- Downey
- 1999
(Show Context)
Citation Context ...n; initial experience indicates many packets are required for inference, leading to either high load of measurement traffic or long measurement intervals, although adaptive approaches can reduce this =-=[9]-=-. More broadly, measurement approaches based on TTL expiry require the cooperation of network elements in returning Internet Control Message Protocol (ICMP) messages. Finally, the success of active me... |

213 | Packet Loss Correlation in the Mbone Multicast Network”, Proceeding of Global Telecommunication Conference
- Yajnik, Kurose, et al.
- 1996
(Show Context)
Citation Context ...ical work on inference methodologies, there has been some ad hoc, statistically non-rigorous work on deriving link-level loss behavior from end-to-end multicast measurements. An estimator proposed in =-=[31]-=- attributes the absence of a packet at a set of receivers to loss on the common path from the source. However, this is biased, even as the number of probes goes to infinity. Some analytic methods for ... |

208 |
Internet tomography: estimating source-destination traffic intensities from link data
- Vardi
- 1996
(Show Context)
Citation Context ...n the common path from the source. However, this is biased, even as the number of probes goes to infinity. Some analytic methods for inference of traffic intensities have been proposed quite recently =-=[28, 29]-=-. The focus of these studies was to determine the intensities of individual source-destination flows from measurements of aggregate flows taken at a number of points in a network. Although there are f... |

172 | Automated packet trace analysis of tcp implementations
- Paxson
- 1997
(Show Context)
Citation Context ...f end-to-end performance is not completely understood. Regarding the second approach, there has been much recent experimental work to understand the phenomenology of end-to-end performance (e.g., see =-=[1, 2, 13, 18, 20, 21]-=-). A number of ongoing measurement infrastructure projects (Felix [5], IPMA [7], NIMI [12] and Surveyor [26]) aim to collect and analyze endto-end measurements across a mesh of paths between a number ... |

160 |
Pathchar: A tool to infer characteristics of internet paths
- Jacobson
- 1997
(Show Context)
Citation Context ...r of ongoing measurement infrastructure projects (Felix [5], IPMA [7], NIMI [12] and Surveyor [26]) aim to collect and analyze endto-end measurements across a mesh of paths between a number of hosts. =-=[10]-=- is under evaluation as a tool for inferring link-level statistics from end-to-end point-to-point measurements. However, much work remains to be done in this area. Furthermore, these efforts are all u... |

156 | Characterizing End-to-End Packet Delay and Loss in the Internet
- Bolot
- 1993
(Show Context)
Citation Context ...model is dictated by the lack of knowledge of the distribution of link delays in networks. While there is significant prior work on the analysis and characterization of end-to-end delay behavior (see =-=[1]-=-, [23], and [26]), to the best of our knowledge there is no general model for per-link delays. The use of a nonparametric model provides the flexibility to capture broadly different delay distribution... |

142 |
Poisson Arrivals See Time Averages
- Wolff
- 1982
(Show Context)
Citation Context ...of network states which are observed via the probes. Probes with exponentially distributed spacings will see time 18averages; this is the PASTA property (Poisson Arrivals See Time Averages; see e.g. =-=[30]-=-). This approach has been proposed for network measurements [22] and is under consideration in the IP Performance Metrics working group of the IETF [8]. In the context of the above discussion, lengthe... |

126 | Estimation and removal of clock skew from network delay measurements
- Moon, Skelley, et al.
- 1999
(Show Context)
Citation Context ...fy the variable component. This can be done provided that a sufficient number of packets incur the minimum one-way delay. Link utilizations are low enough that this appears to be generally true [28], =-=[21]-=-. Second, it is necessary to accurately measure one-way delays so as to be able to subtract the minimum delay. This is easily done if the measurement hosts are deployed with synchronized clocks. Globa... |

122 | Inferring link loss using striped unicast probes
- Duffield, Presti, et al.
- 2001
(Show Context)
Citation Context ...odologies for characterizing link-level behavior from end-to-end measurements. In particular, methods to extend the inference techniques to unicast measurements have been recently proposed in [6] and =-=[11]-=- for the inference of loss rates and [7] and [10] for delay distributions1 . The premise is that unicast measurements could be used to complement multicast measurements for those portions of the netwo... |

99 | The Case for FEC-Based Error Control for Packet Audio in the Internet
- Bolot, Garcia
(Show Context)
Citation Context ...f end-to-end performance is not completely understood. Regarding the second approach, there has been much recent experimental work to understand the phenomenology of end-to-end performance (e.g., see =-=[1, 2, 13, 18, 20, 21]-=-). A number of ongoing measurement infrastructure projects (Felix [5], IPMA [7], NIMI [12] and Surveyor [26]) aim to collect and analyze endto-end measurements across a mesh of paths between a number ... |

83 | On the Dynamics and Significance of Low Frequency Components of Internet Load
- Mukherjee
- 1992
(Show Context)
Citation Context ... is dictated by the lack of knowledge of the distribution of link delays in networks. While there is significant prior work on the analysis and characterization of end-to-end delay behavior (see [1], =-=[23]-=-, and [26]), to the best of our knowledge there is no general model for per-link delays. The use of a nonparametric model provides the flexibility to capture broadly different delay distributions, alb... |

82 | Network loss inference using unicast end-to-end measurement
- Coates, Nowak
- 2000
(Show Context)
Citation Context ... in methodologies for characterizing link-level behavior from end-to-end measurements. In particular, methods to extend the inference techniques to unicast measurements have been recently proposed in =-=[6]-=- and [11] for the inference of loss rates and [7] and [10] for delay distributions1 . The premise is that unicast measurements could be used to complement multicast measurements for those portions of ... |

65 |
Diagnosing Internet Congestion with a Transport Layer Performance Tool
- Mathis, Mahdavi
- 1996
(Show Context)
Citation Context |

62 | Organizing multicast receivers deterministically according to packet-loss correlation
- Levine, Paul, et al.
- 1998
(Show Context)
Citation Context ...rom a multicast source to a receiver, along with other information about that path such as per-hop loss and delay statistics. Topology discovery through mtrace is performed as part of the tracer tool =-=[12]-=-. However, mtrace suffers from performance and applicability problems in the context of large-scale measurements. First, mtrace traces the path from the source to a single receiver by working back thr... |

54 |
Assessing the accuracy of the maximum likelihood estimator: observed versus expected Fisher information
- Efron, Hinkley
- 1978
(Show Context)
Citation Context ...r Confidence Intervals for Parameters. With slight modification, the same methodology can be used to obtain confidence intervals for the parameters " derived from measured data from probes. Following =-=[4]-=- we use the observed Fisher Information: where 1 " / (15) ! Now, the proof of Theorem 2 (see particularly (37)) shows that the depend on the only through the combinations " . Hence the same is true fo... |

52 |
Creating a scalable architecture for internet measurement. http : ==www:psc:edu= mahdavi=nimi paper=NIMI:html
- Adams, Mahdavi, et al.
- 1998
(Show Context)
Citation Context ...cent experimental work to understand the phenomenology of end-to-end performance (e.g., see [1, 2, 13, 18, 20, 21]). A number of ongoing measurement infrastructure projects (Felix [5], IPMA [7], NIMI =-=[12]-=- and Surveyor [26]) aim to collect and analyze endto-end measurements across a mesh of paths between a number of hosts. [10] is under evaluation as a tool for inferring link-level statistics from end-... |

51 |
End-to-end routing behavior
- Paxson
- 1996
(Show Context)
Citation Context |

34 |
Measuring Bottleneck Link Speed
- Carter, Crovella
- 1996
(Show Context)
Citation Context |

33 | Network delay tomography from end-to-end unicast measurements
- Duffield, Horowitz, et al.
- 2001
(Show Context)
Citation Context ...from end-to-end measurements. In particular, methods to extend the inference techniques to unicast measurements have been recently proposed in [6] and [11] for the inference of loss rates and [7] and =-=[10]-=- for delay distributions1 . The premise is that unicast measurements could be used to complement multicast measurements for those portions of the network which do not support multicast. The approach i... |

32 | Applied Statistics - Sachs - 1984 |

31 | Towards a Framework for Defining Internet Performance Metrics - Paxson - 1996 |

19 | Impromptu measurement infrastructures using RTP
- Caceres, Duffield, et al.
- 2002
(Show Context)
Citation Context ...erence has been deployed on the NIMI measurement infrastructure comprising a number of hosts (approximately 50 as of mid–2001). Another approach to multicast measurement has been recently proposed in =-=[2]-=-. This approach, which does not require specialized end hosts, relies on the use the Real-Time Transport Protocol (RTP) and its control protocol (RTCP) [31]. RTP is used to carry multicast audio and v... |

18 | An em approach to od matrix estimation
- Vanderbei, Iannone
- 1994
(Show Context)
Citation Context ...n the common path from the source. However, this is biased, even as the number of probes goes to infinity. Some analytic methods for inference of traffic intensities have been proposed quite recently =-=[28, 29]-=-. The focus of these studies was to determine the intensities of individual source-destination flows from measurements of aggregate flows taken at a number of points in a network. Although there are f... |

16 |
Inference of Multicast Routing Tree Topologies and Bottleneck Bandwidths using End-to-end Measurements
- Ratnasamy, McCanne
- 1999
(Show Context)
Citation Context ...isabled. An alternative approach that is closely related to the work on multicast-based loss inference [3], [4] is to infer the logical multicast topology directly from measured probe statistics; see =-=[29]-=- and [5]. This method does not require cooperation from the network. D. Related Work on Unicast Measurements There has been increasing interest in methodologies for characterizing link-level behavior ... |

16 | Loss-Based Inference of Multicast Network Topology
- Caceres, Duffield, et al.
- 1999
(Show Context)
Citation Context ...An alternative approach that is closely related to the work on multicast-based loss inference [3], [4] is to infer the logical multicast topology directly from measured probe statistics; see [29] and =-=[5]-=-. This method does not require cooperation from the network. D. Related Work on Unicast Measurements There has been increasing interest in methodologies for characterizing link-level behavior from end... |

12 |
Discrete Mathematics
- Ross, Wright
- 1985
(Show Context)
Citation Context ...1] = P[Ω(k) | X f(k) = 1]/P[Xk = 1 | X f(k) = 1] = βk/αk (28) For n ≤ #d(k) let dn(k) denote the set of subsets of d(k) of cardinality n. By the Inclusion-Exclusion Principle (see e.g. Chapter 5.2 of =-=[25]-=-) P[Ω(k)] = P [ ∪ j∈d(k)Ω(j) ] = from which we find using (27) and (28) that #d(k) ∑ ξk = n=1 #d(k) ∑ (−1) n+1 n=1 (−1) n+1 ∑ {j1,...,jn}⊂dn(k) ∑ {j1,...,jn}⊂dn(k) Reexpressed in term of the γk we obt... |

11 |
internet packet dynamics
- End-to-end
- 1997
(Show Context)
Citation Context ...ed by the lack of knowledge of the distribution of link delays in networks. While there is significant prior work on the analysis and characterization of end-to-end delay behavior (see [1], [23], and =-=[26]-=-), to the best of our knowledge there is no general model for per-link delays. The use of a nonparametric model provides the flexibility to capture broadly different delay distributions, albeit at the... |

9 |
Independent Monitoring for Network Survivability. For more information see ßbà áKâ ã-ã ßbà á ä å æ ç/çpè é8ê æ-ä è éˆëjãìá í‹åªã8ëmîDï ã ßbæ ç/ðòñ ã8ð/ó ô‹æHñ&ä õ à$ëËç
- Felix
(Show Context)
Citation Context ...ere has been much recent experimental work to understand the phenomenology of end-to-end performance (e.g., see [1, 2, 13, 18, 20, 21]). A number of ongoing measurement infrastructure projects (Felix =-=[5]-=-, IPMA [7], NIMI [12] and Surveyor [26]) aim to collect and analyze endto-end measurements across a mesh of paths between a number of hosts. [10] is under evaluation as a tool for inferring link-level... |

6 | For more information see http://io.advanced.org/surveyor - Surveyor |

3 | A.: "The Case for FEC-based Error Control for Packet Audio in the Internet - Bolot, Vega-Garcia - 1999 |

1 |
Performance Metrics Working Group. For more information see http://www.ietf.org/html.charters/ippmcharter.html
- IP
(Show Context)
Citation Context ...y (Poisson Arrivals See Time Averages; see e.g. [30]). This approach has been proposed for network measurements [22] and is under consideration in the IP Performance Metrics working group of the IETF =-=[8]-=-. In the context of the above discussion, lengthening the interprobe time is to be understood as increasing the mean of the exponential distribution. 7 Summary and Future Work In this paper, we introd... |

1 |
Theory of Point Estimation”. Wiley-Interscience
- Lehmann
- 1983
(Show Context)
Citation Context ...ions, which are satisfied in the present situation, MLEs exhibit many desirable properties, including strong consistency, asymptotic normality, asymptotic unbiasedness, and asymptotic efficiency (see =-=[11]-=-). Strong consistency means that MLEs converge almost surely (with probability 1) to their target parameters as the sample size increases. The last three properties mean that, if the sample size is la... |

1 | Theory of point estimation". Wiley-Interscience - Lehmann - 1983 |

1 |
Network Animator. For more information see
- unknown authors
(Show Context)
Citation Context ...r simulations, multicast probes are often lost in groups as they compete for queue space with TCP bursts. This phenomenon is readily apparent when watching animations of our simulations with the tool =-=[16]-=-. Inspection of the autocorrelation function of the time series of packet losses for a series of experiments predominantly showed correlations indistinguishable from zero beyond a lag of 1 (i.e. great... |

1 |
For more information .AX=X
- Simulator
(Show Context)
Citation Context ...re described by time-invariant Bernoulli processes. Here we find rapid convergence of the estimates to their actual values as the number of probes increases. The second type of experiment is based on =-=[17]-=- simulations where losses are due to queue overflows as probe traffic competes with other traffic generated by infinite data sources that use the Transmission Control Protocol (TCP) [23]. In this case... |