## Internet Tomography (2002)

Venue: | IEEE Signal Processing Magazine |

Citations: | 116 - 13 self |

### BibTeX

@ARTICLE{Coates02internettomography,

author = {Mark Coates and Alfred Hero and Robert Nowak and Bin Yu},

title = {Internet Tomography},

journal = {IEEE Signal Processing Magazine},

year = {2002},

volume = {19},

pages = {47--65}

}

### Years of Citing Articles

### OpenURL

### Abstract

Today's Internet is a massive, distributed network which continues to explode in size as ecommerce and related activities grow. The heterogeneous and largely unregulated structure of the Internet renders tasks such as dynamic routing, optimized service provision, service level verification, and detection of anomalous/malicious behavior increasingly challenging tasks. The problem is compounded by the fact that one cannot rely on the cooperation of individual servers and routers to aid in the collection of network traffic measurements vital for these tasks. In many ways, network monitoring and inference problems bear a strong resemblance to other "inverse problems" in which key aspects of a system are not directly observable. Familiar signal processing problems such as tomographic image reconstruction, system identification, and array processing all have interesting interpretations in the networking context. This article introduces the new field of network tomography, a field which we believe will benefit greatly from the wealth of signal processing theory and algorithms.

### Citations

1119 |
Computer Networks
- Tanenbaum
- 1996
(Show Context)
Citation Context ...e fairly self-contained; only a modest familiarity with networking principles is required and basic concepts are dened as necessary. For more background information, the a number of recent textbooks [=-=15, 16, 17, 18, 19, 20, 21]-=- provide an excellent introductions to theseld of networking. The article is organized as follows. First we brie y review the area of large scale network inference and tomography. We then discuss link... |

735 | End-to-end internet packet dynamics
- Paxson
- 1999
(Show Context)
Citation Context ...g accurate estimates ofsne grain network behavior, e.g., the dynamics of node trac rates, service times, and queue lengths. The area of statistical modeling of network trac is a mature and activeseld =-=[8, 9, 10, 11, 12]-=-. Sophisticated fractal and multifractal models of single trac streams can account for long range dependency, non-Gaussian distributions, and other peculiar behaviors. Such self similar behavior of tr... |

607 |
Multivariate Density Estimation. Theory, Practice and Visualization
- Scott
- 1992
(Show Context)
Citation Context ... � error norm is simply the sum of the absolute difference between the estimated pmf and the true pmf over the K bins (MLE estimates made with fewer bins are appropriately converted). As discussed in =-=[15]-=-, the� � error criterion is a common measure of the performance of a density estimate. The advantage of such a measure as opposed to a mean-squared error criterion is that more attention is paid to th... |

503 |
Computer Networking: a top-down approach featuring the Internet
- Kurose, Ross
- 2001
(Show Context)
Citation Context ...e fairly self-contained; only a modest familiarity with networking principles is required and basic concepts are dened as necessary. For more background information, the a number of recent textbooks [=-=15, 16, 17, 18, 19, 20, 21]-=- provide an excellent introductions to theseld of networking. The article is organized as follows. First we brie y review the area of large scale network inference and tomography. We then discuss link... |

429 |
The mathematics of computerized tomography
- Natterer
- 1986
(Show Context)
Citation Context ...sional problems of estimating are specic examples of inverse problems. Inverse problems have a very extensive literature both in signal processing [37], statistics [38], and in applied mathematics [3=-=9-=-]. Solution methods for such inverse problems depend on the nature of the noise and the A matrix and typically require iterative algorithms since they cannot be solved directly. In general, A is not ... |

344 |
Computer Networks, a Systems Approach
- Peterson, Davie
- 1996
(Show Context)
Citation Context ...e fairly self-contained; only a modest familiarity with networking principles is required and basic concepts are dened as necessary. For more background information, the a number of recent textbooks [=-=15, 16, 17, 18, 19, 20, 21]-=- provide an excellent introductions to theseld of networking. The article is organized as follows. First we brie y review the area of large scale network inference and tomography. We then discuss link... |

343 |
End-to-end packet delay and loss behavior in the internet
- Bolot
- 1993
(Show Context)
Citation Context ...t the second packet was successfully received suggests that the queue for link j was not full when thesrst packet arrived. Evidence for such behavior has been provided by observations of the Internet =-=[55, 51-=-]. Denote the complete set of conditional success probabilities bys fsj g m j=1 . Suppose that each sender sends a large number of back-to-back packet pairs in which thesrst packet is destined for one... |

325 |
An Engineering Approach to Computer Networking
- Keshav
(Show Context)
Citation Context |

296 | Routing in the Internet
- Huitema
- 1995
(Show Context)
Citation Context |

269 |
Graphical models for machine learning and digital communication
- Frey
- 1998
(Show Context)
Citation Context ...log likelihood (3) is linear in� . Thus, in the E-Step we need only compute the exof� ��� � ���� � pectation . This can be efficiently performed using a message passing (or upward-downward) procedure =-=[14]-=-. Unfortunately, a straightforward implementation of the message passing procedure, as proposed in [2, 3], has a computational complexity is��¤�� ��� § which where� , is the number of links in the tre... |

263 | Multicast-based inference of network-internal delay distributions
- Presti, Duffield, et al.
- 2002
(Show Context)
Citation Context ...rk inference and medical tomography. Two forms of network tomography have been addressed in the recent literature: i) link-level parameter estimation based on end-to-end, path-level trac measurements =-=[23, 24, 25, 26, 27, 28, 29, 30, 31, 32]-=- and ii) sender-receiver path-level trac intensity estimation based on link-level trac measurements [33, 22, 34, 35, 36]. In link-level parameter estimation, the trac measurements typically consist of... |

263 |
I -divergence geometry of probability distributions and minimization problems
- Csiszár
- 1975
(Show Context)
Citation Context ...parameters, and the positivity constraints on the OD trac, are used as the initial estimates of the OD trac. The linear equation y = Ax is enforced via the iterative proportionalstting algorithm (cf. =-=[59, 60]-=-) to obtain thesnal estimates of the OD trac. The positivity and the linear constraints are very importantsnal steps to get reliable estimates of the OD trac, in addition to the implicit regularizatio... |

219 |
Network tomography: estimating source-destination traffic intensities from link data
- Vardi
- 1996
(Show Context)
Citation Context ...network performance parameters based on trac measurements at a limited subset of the nodes. Y. Vardi was one of thesrst to rigorously study this sort of problem and coined the term network tomography =-=[22]-=- due to the similarity between network inference and medical tomography. Two forms of network tomography have been addressed in the recent literature: i) link-level parameter estimation based on end-t... |

196 | Measuring Link Bandwidths Using a Deterministic Model of Packet Delay
- Lai, Baker
- 2000
(Show Context)
Citation Context ...hen the same methodology developed for multicast-based tomography (as described above) can be employed with unicast, packet-pair measurements [27]. In the case of bandwidth tomography, the authors of =-=[52]-=- addressed the challenge of nonuniqueness through clever use of the headerselds of unicast packets. The time-to-live (TTL)seld in each packet header indicates how many hops the packet should traverse.... |

181 | A Multifractal Wavelet Model with Application to TCP Network Traffic
- Riedi, Crouse, et al.
- 1999
(Show Context)
Citation Context ...g accurate estimates ofsne grain network behavior, e.g., the dynamics of node trac rates, service times, and queue lengths. The area of statistical modeling of network trac is a mature and activeseld =-=[8, 9, 10, 11, 12]-=-. Sophisticated fractal and multifractal models of single trac streams can account for long range dependency, non-Gaussian distributions, and other peculiar behaviors. Such self similar behavior of tr... |

160 |
Local regression and likelihood
- Loader
- 1999
(Show Context)
Citation Context ...ability of the parameters up to the scale parameter which can be determined from the expectation of a y. Cao et al. [34] address the non-stationarity in the data using a local likelihood model (cf. [=-=58]-=-); that is, for any given time t, analysis is based on a likelihood function derived from the observations within a symmetric window of size w = 2h + 1 around t (e.g., in the experiments described bel... |

146 | Detecting Shared Congestion of Flows via End-to-end Measurement
- Rubenstein, Kurose, et al.
- 2000
(Show Context)
Citation Context ...t all internal devices for a number of reasons [1]. This has prompted several groups to investigate methods for inferring internal network behavior based on “external” end-to-end network measurements =-=[1, 2, 3, 4, 5, 6, 7, 8]-=-. This problem is often referred to as network tomography. Queuing delays are one of the most critical performance characteristics. Optimizing communication network routing and service strategies requ... |

125 | Inferring Link Loss Using Striped Unicast Probes
- Duffield, Presti, et al.
- 2001
(Show Context)
Citation Context ...rk inference and medical tomography. Two forms of network tomography have been addressed in the recent literature: i) link-level parameter estimation based on end-to-end, path-level trac measurements =-=[23, 24, 25, 26, 27, 28, 29, 30, 31, 32]-=- and ii) sender-receiver path-level trac intensity estimation based on link-level trac measurements [33, 22, 34, 35, 36]. In link-level parameter estimation, the trac measurements typically consist of... |

108 |
A statistical perspective on ill-posed inverse problems (with discussion
- O’Sullivan
- 1986
(Show Context)
Citation Context ...ternet. The associated high dimensional problems of estimating are specic examples of inverse problems. Inverse problems have a very extensive literature both in signal processing [37], statistics [3=-=8-=-], and in applied mathematics [39]. Solution methods for such inverse problems depend on the nature of the noise and the A matrix and typically require iterative algorithms since they cannot be solve... |

95 | Time-varying network tomography: router link data
- Cao, Davis, et al.
- 2000
(Show Context)
Citation Context ...stimation based on end-to-end, path-level trac measurements [23, 24, 25, 26, 27, 28, 29, 30, 31, 32] and ii) sender-receiver path-level trac intensity estimation based on link-level trac measurements =-=[33, 22, 34, 35, 36]-=-. In link-level parameter estimation, the trac measurements typically consist of counts of packets transmitted and/or received between nodes or time delays between packet transmissions and receptions.... |

91 | Inference of Multicast Routing Trees and Bottleneck Bandwidths using End-to-End Measurements
- Ratnasamy, McCanne
- 1999
(Show Context)
Citation Context ... not applicable, a number of methods have been proposed for the identication of network (routing) topology based on end18 to-end measurements that measure the degree of correlation between receivers [=-=30, 40, 41, 43, 44, 45]-=-. Most of these approaches have concentrated on identifying the tree structured topology connecting a single sender to multiple receivers. It is assumed that the routes from the sender to the receiver... |

84 | Network Loss Inference Using Unicast End-to-End Measurement
- Coates, Nowak
- 2000
(Show Context)
Citation Context ... log success probability associated with each link. Clearly, a unique solution for f j g does not exist since A is not full rank. To address this challenge in unicast loss tomography, the authors of [=-=25]-=- and [28] independently proposed methodologies based on measurements made using unicast, back-to-back packet pairs. These measurements provide an opportunity to collect more informative statistics tha... |

77 | Maximum likelihood network topology identification from edge-based unicast measurements
- Coates, Castro, et al.
- 2002
(Show Context)
Citation Context ...ing methods [34, 26]. Another variation on the basic problem (1) is obtained by assuming that the routing matrix A is not known precisely. This leads to the so-called \topology identication" prob=-=lem [30, 40, 41, 42, 43, 44, 45-=-], and is somewhat akin to blind deconvolution or system identication problems. 3 Link-Level Network Inference Link-level network tomography is the estimation of link-level network parameters (loss ra... |

72 |
On the Self-Similar Nature of Ethernet Trac (Extended Version
- Leland, Taqqu, et al.
- 1994
(Show Context)
Citation Context ...g accurate estimates ofsne grain network behavior, e.g., the dynamics of node trac rates, service times, and queue lengths. The area of statistical modeling of network trac is a mature and activeseld =-=[8, 9, 10, 11, 12]-=-. Sophisticated fractal and multifractal models of single trac streams can account for long range dependency, non-Gaussian distributions, and other peculiar behaviors. Such self similar behavior of tr... |

70 | Multicast inference of packet delay variance at interior network links
- Duffield, Presti
- 2000
(Show Context)
Citation Context ... : (4) A least squares estimate of f i g is easily computed for this overdetermined system of equations. Sophisticated and eective algorithms have been derived for large-scale network tomography in [2=-=4, 48, 25, 49]-=-. Similar procedures can be conducted in the case of delay distribution tomography. There is a certain minimum propagation delay along each link, which is assumed known. Multicast a packet from node 0... |

67 | Robust identification of shared losses using end-to-end unicast probes
- Harfoush, Bestavros, et al.
- 2000
(Show Context)
Citation Context ...t all internal devices for a number of reasons [1]. This has prompted several groups to investigate methods for inferring internal network behavior based on “external” end-to-end network measurements =-=[1, 2, 3, 4, 5, 6, 7, 8]-=-. This problem is often referred to as network tomography. Queuing delays are one of the most critical performance characteristics. Optimizing communication network routing and service strategies requ... |

66 | Multicast topology inference from measured end-to-end loss
- Duffield, Horowitz, et al.
(Show Context)
Citation Context ...ing methods [34, 26]. Another variation on the basic problem (1) is obtained by assuming that the routing matrix A is not known precisely. This leads to the so-called \topology identication" prob=-=lem [30, 40, 41, 42, 43, 44, 45-=-], and is somewhat akin to blind deconvolution or system identication problems. 3 Link-Level Network Inference Link-level network tomography is the estimation of link-level network parameters (loss ra... |

47 | Scaling analysis of conservative cascades, with applications to network traffic
- Gilbert, Willinger, et al.
- 1999
(Show Context)
Citation Context |

43 | Inference and labeling of metric-induced network topologies
- Bestavros, Byers, et al.
- 2002
(Show Context)
Citation Context ...ing methods [34, 26]. Another variation on the basic problem (1) is obtained by assuming that the routing matrix A is not known precisely. This leads to the so-called \topology identication" prob=-=lem [30, 40, 41, 42, 43, 44, 45-=-], and is somewhat akin to blind deconvolution or system identication problems. 3 Link-Level Network Inference Link-level network tomography is the estimation of link-level network parameters (loss ra... |

38 | Passive network tomography using em algorithms
- Tsang, Coates, et al.
(Show Context)
Citation Context ... : (4) A least squares estimate of f i g is easily computed for this overdetermined system of equations. Sophisticated and eective algorithms have been derived for large-scale network tomography in [2=-=4, 48, 25, 49]-=-. Similar procedures can be conducted in the case of delay distribution tomography. There is a certain minimum propagation delay along each link, which is assumed known. Multicast a packet from node 0... |

35 |
On the propagation of long-range dependence
- Kenesi, Vattay
- 2000
(Show Context)
Citation Context ... streams can account for long range dependency, non-Gaussian distributions, and other peculiar behaviors. Such self similar behavior of trac rates has been validated for heavily loaded wired networks =-=[13]-=-. For a detailed overview of these and other statistical trac models we refer the reader to the companion article(s) in this special issue. To date these models are overly complicated to be incorporat... |

30 | Sequential Monte Carlo inference of internal delays in nonstationary data networks
- Coates, Nowak
- 2002
(Show Context)
Citation Context ...odel can be used to simplify the inference process. While some progress has been made on 3 incorporating simplesrst order spatio-temporal dependency models into large scale network inference problems =-=[14-=-] much work remains to be done. This article attempts to be fairly self-contained; only a modest familiarity with networking principles is required and basic concepts are dened as necessary. For more ... |

25 |
Queueing Networks: Customers, Signals, and Product Form Solutions
- Chao, Miyazawa, et al.
- 1999
(Show Context)
Citation Context ...ing mechanisms and network trac behavior, are generally adequate for the inference of gross-level performance characteristics. Focus is shifted from detailed mathematical modeling of network dynamics =-=[3, 4]-=- to careful handling of measurement and probing strategies, large scale computations, and model validation. The measurement methodologies require: software tools for monitoring tracsow and generating ... |

25 | Passive unicast network tomography using EM algorithm
- Tsang, Coates, et al.
- 2001
(Show Context)
Citation Context ...rement, probe packets are sent by the senders specically for the purpose of estimation. In passive monitoring, the sender extracts data from existing communications (e.g., records of TCP 3 sessions) [=-=49, 53]-=-. Loss rate and delay distribution tomography methods have been 2 A droptail queuing policy means that a packet is dropped by a queue only if it reaches the queue and there is insucient space in the b... |

22 | Multicast topology inference from end-to-end measurements
- eld, Horowitz, et al.
- 2000
(Show Context)
Citation Context |

21 |
Network delay distribution inference from end-to-end unicast measurement
- Coates, Nowak
- 2001
(Show Context)
Citation Context ...olution. Several methods for unraveling this convolution from the end-to-end densities are: 1) transformation of the convolution into a more tractable matrix operator via discretization of the delays =-=[29, 26, 31]-=-; 2) estimation of low order moments such as link delay variance [48] from end-to-end delay variances which are additive over the probe paths; 3) nonparametric density estimation methods in combinatio... |

20 | Unicast inference of network link delay distributions from edge measurements
- Shih, Hero
- 2001
(Show Context)
Citation Context ...sed to describe the observed delay measurements. Most work to date in network tomography is based on parametric models. A nonparametric approach based on cumulant generating functions was proposed in =-=[11]-=-, but this approach, unlike ours, requires internal measurements. Parametric models assume that the measured traffic data depends on a finite number of parameters. For example, earlier work in delay d... |

19 |
Willinger: \Dynamics of IP trac: A study of the role of variability and the impact of control
- Feldmann, Gilbert, et al.
- 1999
(Show Context)
Citation Context |

19 | Unicast-based inference of network link delay distributions with finite mixture models
- Shih, Hero
- 2003
(Show Context)
Citation Context ...rk inference and medical tomography. Two forms of network tomography have been addressed in the recent literature: i) link-level parameter estimation based on end-to-end, path-level trac measurements =-=[23, 24, 25, 26, 27, 28, 29, 30, 31, 32]-=- and ii) sender-receiver path-level trac intensity estimation based on link-level trac measurements [33, 22, 34, 35, 36]. In link-level parameter estimation, the trac measurements typically consist of... |

19 | An EM approach to OD matrix estimation
- Vanderbei, Iannone
- 1994
(Show Context)
Citation Context ...stimation based on end-to-end, path-level trac measurements [23, 24, 25, 26, 27, 28, 29, 30, 31, 32] and ii) sender-receiver path-level trac intensity estimation based on link-level trac measurements =-=[33, 22, 34, 35, 36]-=-. In link-level parameter estimation, the trac measurements typically consist of counts of packets transmitted and/or received between nodes or time delays between packet transmissions and receptions.... |

16 |
UNIX Network Programming Vol 1: Networking APIs – The Sockets Networking API, volume 1 of 3
- Stevens
- 2004
(Show Context)
Citation Context |

10 |
Networks for networks: Internet analysis using Bayesian graphical models
- Coates, Nowak
- 2000
(Show Context)
Citation Context ...k-internal Characteristics (MINC) Project at the University of Massachusetts [23] pioneered the use of multicast probing for 7 network tomography, and stimulated much of the current work in this area =-=[23, 24, 25, 46, 26, 27, 29, 30, 31, 47, 32]-=-. We now outline a general framework for the link-level tomography problems. Consider network depicted in Figure 2(a). This illustrates the scenario where packets are sent from a set of sources to a n... |

8 |
Estimation of network link loss rates via chaining in multicast trees
- Ziotopolous, Hero, et al.
- 2001
(Show Context)
Citation Context ...e indirectly related to the decision-making objectives of the end-user regarding the existence of anomalous network conditions. An example of this is bottleneck detection which has been considered in =-=[47, 32]-=- as an application of reconstructed delay or loss estimation. However, systematic development of large scale hypothesis testing theory for networks would undoubtedly lead to superior detection perform... |

7 |
A multiresolution analysis for likelihoods: Theory and methods," Article provided by Nowak
- Kolaczyk, Nowak
(Show Context)
Citation Context ... number of non-zero Haar wavelet coefficients where� in the delay pmf of link¢. This wavelet-based scheme is called a Multiscale Maximum Penalized Likelihood Estimator (MMPLE), and it was proposed in =-=[13]-=- for conventional (non-tomographic) probability density estimation. It penalizes the irregularity and complexity of the pmf — the larger the value � � of , the more “bumps” in the pmf. There are two i... |

6 |
Robust Identi of Shared Losses Using End-to-End Unicast Probes
- Harfoush, Bestavros, et al.
- 2000
(Show Context)
Citation Context ...the success probabilities on individual links (between routers) in the path. To overcome this diculty, measurements are made using back-to-back packet pairs or sequences of packets as discussed above =-=[25, 2-=-8, 27]. If two back-to-back packets are sent to node j from its parent node (j), then dene the conditional success probability assj Pr(1st packet (j) ! j j 2nd packet (j) ! j ); where (j) ! j is shor... |

5 |
Deriving trac demands for operational IP networks: methodology and experience
- Feldmann, Greenberg, et al.
- 2000
(Show Context)
Citation Context ...direct methods collect sums of OD trac counts and are considered in [22, 33, 36, 34]; a direct method to measure OD trac counts via software such as NetFlow supported by Cisco routers is described in =-=[34, 57]-=-. Both approaches need the cooperation of the routers in the network, but this is not problematic for privately-owned networks. The link trac counts at routers are much easier to collect relative to t... |

4 |
Stochastic networks: theory and applications. Royal Statistical Society Lecture Note Series
- Kelly, Zachary, et al.
- 1996
(Show Context)
Citation Context ...ing mechanisms and network trac behavior, are generally adequate for the inference of gross-level performance characteristics. Focus is shifted from detailed mathematical modeling of network dynamics =-=[3, 4]-=- to careful handling of measurement and probing strategies, large scale computations, and model validation. The measurement methodologies require: software tools for monitoring tracsow and generating ... |

4 |
Trajectory sampling for direct tra#c observation
- Dueld, Grosslauer
- 2000
(Show Context)
Citation Context ...te measure and report packet transport attributes of the round-trip path (from sender to receiver and back) of a probe (see [1] for a survey of these and other measurement tools). Trajectory sampling =-=[7]-=- is another example of an active probing software tool. These methods depend on accurate reporting by all nodes along the route and many require special assumptions, e.g., symmetric forward/reverse li... |

3 |
Inverse problems and signal processing
- Mammone
- 1998
(Show Context)
Citation Context ...s sites in the Internet. The associated high dimensional problems of estimating are specic examples of inverse problems. Inverse problems have a very extensive literature both in signal processing [3=-=7-=-], statistics [38], and in applied mathematics [39]. Solution methods for such inverse problems depend on the nature of the noise and the A matrix and typically require iterative algorithms since the... |

2 |
Bayesian inference on network trac using link count data
- Tebaldi, West
- 1998
(Show Context)
Citation Context ...stimation based on end-to-end, path-level trac measurements [23, 24, 25, 26, 27, 28, 29, 30, 31, 32] and ii) sender-receiver path-level trac intensity estimation based on link-level trac measurements =-=[33, 22, 34, 35, 36]-=-. In link-level parameter estimation, the trac measurements typically consist of counts of packets transmitted and/or received between nodes or time delays between packet transmissions and receptions.... |