Results 1  10
of
169
Modeling TCP Throughput: A Simple Model and its Empirical Validation
, 1998
"... In this paper we develop a simple analytic characterization of the steady state throughput, as a function of loss rate and round trip time for a bulk transfer TCP flow, i.e., a flow with an unlimited amount of data to send. Unlike the models in [6, 7, 10], our model captures not only the behavior of ..."
Abstract

Cited by 1109 (37 self)
 Add to MetaCart
In this paper we develop a simple analytic characterization of the steady state throughput, as a function of loss rate and round trip time for a bulk transfer TCP flow, i.e., a flow with an unlimited amount of data to send. Unlike the models in [6, 7, 10], our model captures not only the behavior of TCP’s fast retransmit mechanism (which is also considered in [6, 7, 10]) but also the effect of TCP’s timeout mechanism on throughput. Our measurements suggest that this latter behavior is important from a modeling perspective, as almost all of our TCP traces contained more timeout events than fast retransmit events. Our measurements demonstrate that our model is able to more accurately predict TCP throughput and is accurate over a wider range of loss rates. This material is based upon work supported by the National Science Foundation under grants NCR9508274, NCR9523807 and CDA9502639. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Convergence of Stochastic Iterative Dynamic Programming Algorithms
 Neural Computation
, 1994
"... Increasing attention has recently been paid to algorithms based on dynamic programming (DP) due to the suitability of DP for learning problems involving control. In stochastic environments where the system being controlled is only incompletely known, however, a unifying theoretical account of th ..."
Abstract

Cited by 205 (8 self)
 Add to MetaCart
Increasing attention has recently been paid to algorithms based on dynamic programming (DP) due to the suitability of DP for learning problems involving control. In stochastic environments where the system being controlled is only incompletely known, however, a unifying theoretical account of the behavior of these methods has been missing. In this paper we relate DPbased learning algorithms to powerful techniques of stochastic approximation via a new convergence theorem, enabling us to establish a class of convergent algorithms to which both TD() and Qlearning belong. 1
On Optimal Call Admission Control in Cellular Networks
 Wireless Networks
, 1996
"... Two important QualityofService (QoS) measures for current cellular networks are the fractions of new and handoff "calls" that are blocked due to unavailability of "channels" (radio and/or computing resources). Based on these QoS measures, we derive optimal admission control policies for three prob ..."
Abstract

Cited by 126 (2 self)
 Add to MetaCart
Two important QualityofService (QoS) measures for current cellular networks are the fractions of new and handoff "calls" that are blocked due to unavailability of "channels" (radio and/or computing resources). Based on these QoS measures, we derive optimal admission control policies for three problems: minimizing a linear objective function of the new and handoff call blocking probabilities (MINOBJ), minimizing the new call blocking probability with a hard constraint on the handoff call blocking probability (MINBLOCK) and minimizing the number of channels with hard constraints on both of the blocking probabilities (MINC). We show that the wellknown Guard Channel policy is optimal for the MINOBJ problem, while a new Fractional Guard Channel policy is optimal for the MINBLOCK and MINC problems. The Guard Channel policy reserves a set of channels for handoff calls while the Fractional Guard Channel policy effectively reserves a nonintegral number of guard channels for handoff calls by...
Bayesian Estimation Of Motion Vector Fields
 IEEE Trans. Pattern Anal. Machine Intell
, 1992
"... This paper presents a new approach to the estimation of twodimensional motion vector fields from timevarying images. The approach is stochastic, both in its formulation and in the solution method. The formulation involves the specification of a deterministic structural model, along with stochastic ..."
Abstract

Cited by 121 (19 self)
 Add to MetaCart
This paper presents a new approach to the estimation of twodimensional motion vector fields from timevarying images. The approach is stochastic, both in its formulation and in the solution method. The formulation involves the specification of a deterministic structural model, along with stochastic observation and motion field models. Two motion models are proposed: a globally smooth model based on vector Markov random fields and a piecewise smooth model derived from coupled vectorbinary Markov random fields. Two estimation criteria are studied. In the Maximum A Posteriori Probability (MAP) estimation the a posteriori probability of motion given data is maximized, while in the Minimum Expected Cost (MEC) estimation the expectation of a certain cost function is minimized. The MAP estimation is performed via simulated annealing, while the MEC algorithm performs iterationwise averaging. Both algorithms generate sample fields by means of stochastic relaxation implemented via the Gibbs s...
Reinforcement Learning Methods for ContinuousTime Markov Decision Problems
 Advances in Neural Information Processing Systems
, 1994
"... SemiMarkov Decision Problems are continuous time generalizations of discrete time Markov Decision Problems. A number of reinforcement learning algorithms have been developed recently for the solution of Markov Decision Problems, based on the ideas of asynchronous dynamic programming and stochastic ..."
Abstract

Cited by 113 (0 self)
 Add to MetaCart
SemiMarkov Decision Problems are continuous time generalizations of discrete time Markov Decision Problems. A number of reinforcement learning algorithms have been developed recently for the solution of Markov Decision Problems, based on the ideas of asynchronous dynamic programming and stochastic approximation. Among these are TD(), Qlearning, and Realtime Dynamic Programming. After reviewing semiMarkov Decision Problems and Bellman's optimality equation in that context, we propose algorithms similar to those named above, adapted to the solution of semiMarkov Decision Problems. We demonstrate these algorithms by applying them to the problem of determining the optimal control for a simple queueing system. We conclude with a discussion of circumstances under which these algorithms may be usefully applied. 1 Introduction A number of reinforcement learning algorithms based on the ideas of asynchronous dynamic programming and stochastic approximation have been developed recently for...
Building LowDiameter P2P Networks
, 2001
"... In a peertopeer (P2P) network, nodes connect into an existing network and participate in providing and availing of services. There is no dichotomy between a central server and distributed clients. Current P2P networks (e.g., Gnutella) are constructed by participants following their own uncoordina ..."
Abstract

Cited by 108 (2 self)
 Add to MetaCart
In a peertopeer (P2P) network, nodes connect into an existing network and participate in providing and availing of services. There is no dichotomy between a central server and distributed clients. Current P2P networks (e.g., Gnutella) are constructed by participants following their own uncoordinated (and often whimsical) protocols; they consequently suffer from frequent network overload and fragmentation into disconnected pieces separated by chokepoints with inadequate bandwidth. In this paper we propose a simple scheme for participants to build P2P networks in a distributed fashion, and prove that it results in connected networks of constant degree and logarithmic diameter. It does so with no global knowledge of all the nodes in the network. In the most common P2P application to date (search), these properties are crucial.
Consideration of Risk in Reinforcement Learning
, 1994
"... Most Reinforcement Learning (RL) work supposes policies for sequential decision tasks to be optimal that minimize the expected total discounted cost (e. g. Q Learning [Wat 89], AHC [Bar Sut And 83]). On the other hand, it is well known that it is not always reliable and can be treacherous to use t ..."
Abstract

Cited by 50 (0 self)
 Add to MetaCart
Most Reinforcement Learning (RL) work supposes policies for sequential decision tasks to be optimal that minimize the expected total discounted cost (e. g. Q Learning [Wat 89], AHC [Bar Sut And 83]). On the other hand, it is well known that it is not always reliable and can be treacherous to use the expected value as a decision criterion [Tha 87]. A lot of alternative decision criteria have been suggested in decision theory to get a more sophisticated considaration of risk but most RL researchers have not concerned themselves with this subject until now. The purpose of this paper is to draw the reader's attention to the problems of the expected value criterion in Markov Decision Processes and to give Dynamic Programming algorithms for an alternative criterion, namely the Minimax criterion. A counterpart to Watkins' QLearning related to the Minimax criterion is presented. The new algorithm, called $ Q  Learning , finds policies that minimize the worstcase total discounted costs....
Duality And Linear Programs For Stability And Performance Analysis Of Queueing Networks And Scheduling Policies
 IEEE Transactions on Automatic Control
, 1996
"... We consider the problems of performance analysis and stability/instability determination of queueing networks and scheduling policies. We exhibit a strong duality relationship between the performance of a system, and its stability analysis via mean drift. We obtain a variety of linear programs to co ..."
Abstract

Cited by 48 (28 self)
 Add to MetaCart
We consider the problems of performance analysis and stability/instability determination of queueing networks and scheduling policies. We exhibit a strong duality relationship between the performance of a system, and its stability analysis via mean drift. We obtain a variety of linear programs to conduct such stability and performance analyses. A certain LP, called the Performance LP, bounds the performance of all stationary nonidling scheduling policies. If it is bounded, then its dual, called the Drift LP, has a feasible solution, which is a copositive matrix. The quadratic form associated with this copositive matrix has a negative drift, allowing us to conclude that all stationary nonidling scheduling policies are stable in the very strong sense of having a geometrically converging exponential moment. Some systems satisfy an auxiliary set of linear constraints. Examples are systems operating under some special scheduling policies such as buffer priority policies, or systems incorp...
Predicting Resource Usage and Estimation Accuracy in an IP Flow Measurement Collection Infrastructure
, 2003
"... This paper describes a measurement infrastructure used to collect detailed IP traffic measurements from an IP backbone. Usage, i.e, bytes transmitted, is determined from raw NetFlow records generated by the backbone routers. The amount of raw data is immense. Two types of data sampling in order to m ..."
Abstract

Cited by 42 (8 self)
 Add to MetaCart
This paper describes a measurement infrastructure used to collect detailed IP traffic measurements from an IP backbone. Usage, i.e, bytes transmitted, is determined from raw NetFlow records generated by the backbone routers. The amount of raw data is immense. Two types of data sampling in order to manage data volumes: (i) (packet) sampled NetFlow in the routers; (ii) sizedependent sampling of NetFlow records. Furthermore, dropping of NetFlow records in transmission can be regarded as an uncontrolled form of sampling.
Building LowDiameter PeertoPeer Networks
 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 21, NO. 6, AUGUST 2003
, 2003
"... PeertoPeer (P2P) computing has emerged as a significant paradigm for providing distributed services, in particular search and data sharing. Current P2P networks (e.g., Gnutella) are constructed by participants following their own uncoordinated (and often whimsical) protocols; they consequently suf ..."
Abstract

Cited by 42 (1 self)
 Add to MetaCart
PeertoPeer (P2P) computing has emerged as a significant paradigm for providing distributed services, in particular search and data sharing. Current P2P networks (e.g., Gnutella) are constructed by participants following their own uncoordinated (and often whimsical) protocols; they consequently suffer from frequent network overload and partitioning into disconnected pieces separated by choke points with inadequate bandwidth. In this