Results 1  10
of
14
Restless Bandits, Partial Conservation Laws and Indexability
, 2001
"... We show that if performance measures in a stochastic scheduling problem satisfy a set of socalled partial conservation laws (PCL), which extend previously studied generalized conservation laws (GCL), then the problem is solved optimally by a priorityindex policy for an appropriate range of linear ..."
Abstract

Cited by 33 (9 self)
 Add to MetaCart
We show that if performance measures in a stochastic scheduling problem satisfy a set of socalled partial conservation laws (PCL), which extend previously studied generalized conservation laws (GCL), then the problem is solved optimally by a priorityindex policy for an appropriate range of linear performance objectives, where the optimal indices are computed by a onepass adaptivegreedy algorithm, based on Klimov's. We further apply this framework to investigate the indexability property of restless bandits introduced by Whittle, obtaining the following results: (1) we identify a class of restless bandits (PCLindexable) which are indexable
Validity of heavy traffic steadystate approximations in open queueing networks
, 2006
"... We consider a single class open queueing network, also known as a generalized Jackson network (GJN). A classical result in heavytraffic theory asserts that the sequence of normalized queue length processes of the GJN converge weakly to a reflected Brownian motion (RBM) in the orthant, as the traffic ..."
Abstract

Cited by 28 (4 self)
 Add to MetaCart
We consider a single class open queueing network, also known as a generalized Jackson network (GJN). A classical result in heavytraffic theory asserts that the sequence of normalized queue length processes of the GJN converge weakly to a reflected Brownian motion (RBM) in the orthant, as the traffic intensity approaches unity. However, barring simple instances, it is still not known whether the stationary distribution of RBM provides a valid approximation for the steadystate of the original network. In this paper we resolve this open problem by proving that the rescaled stationary distribution of the GJN converges to the stationary distribution of the RBM, thus validating a socalled “interchangeoflimits” for this class of networks. Our method of proof involves a combination of Lyapunov function techniques, strong approximations and tail probability bounds that yield tightness of the sequence of stationary distributions of the GJN.
Performance of multiclass Markovian queueing networks via piecewise linear Lyapunov functions
 Annals of Applied Probability
, 2001
"... We study the distribution of steadystate queue lengths in multiclass queueing networks under a stable policy. We propose a general methodology based on Lyapunov functions, for the performance analysis of infinite state Markov chains and apply it specifically to Markovian multiclass queueing network ..."
Abstract

Cited by 22 (3 self)
 Add to MetaCart
We study the distribution of steadystate queue lengths in multiclass queueing networks under a stable policy. We propose a general methodology based on Lyapunov functions, for the performance analysis of infinite state Markov chains and apply it specifically to Markovian multiclass queueing networks. We establish a deeper connection between stability and performance of such networks by showing that if there exist linear and piecewise linear Lyapunov functions that show stability, then these Lyapunov functions can be used to establish geometric type lower and upper bounds on the tail probabilities, and thus bounds on the expectation of the queue lengths. As an example of our results, for a reentrant line queueing network with two processing stations operating under a workconserving policy we showthat E[L] =O 1 (1; ) 2 � where L is the total number ofcustomers in the system, and is the maximal actual or virtual traffic intensity inthenetwork. In a Markovian setting, this extends a recent result by Daiand Vande Vate, which states that a reentrant line queueing network with two stations is globally stable if < 1: We also present several results on the
Applications of Markov Decision Processes in Communication Networks: a Survey
 in Markov Decision Processes, Models, Methods, Directions, and Open Problems, E. Feinberg and A. Shwartz (Editors) Kluwer
, 2001
"... We present in this Chapter a survey on applications of MDPs to communication networks. We survey both the different applications areas in communication networks as well as the theoretical tools that have been developed to model and to solve the resulting control problems. 1 ..."
Abstract

Cited by 14 (2 self)
 Add to MetaCart
We present in this Chapter a survey on applications of MDPs to communication networks. We survey both the different applications areas in communication networks as well as the theoretical tools that have been developed to model and to solve the resulting control problems. 1
Moment Problems and Semidefinite Optimization
 WORKING PAPER, SLOAN SCHOOL OF MANAGEMENT, MIT
, 2000
"... Problems involving moments of random variables arise naturally in many areas of mathematics, economics, and operations research. How dowe obtain optimal bounds on the probability that a random variable belongs in a set, given some of its moments? How dowe price financial derivatives without assuming ..."
Abstract

Cited by 11 (0 self)
 Add to MetaCart
Problems involving moments of random variables arise naturally in many areas of mathematics, economics, and operations research. How dowe obtain optimal bounds on the probability that a random variable belongs in a set, given some of its moments? How dowe price financial derivatives without assuming any model for the underlying price dynamics, given only moments of the price of the underlying asset? How do we obtain stronger relaxations for stochastic optimization problems exploiting the knowledge that the decision variables are moments of random variables? Can we generate near optimal solutions for a discrete optimization problem from a semidefinite relaxation by interpreting an optimal solution of the relaxation as a covariance matrix? In this paper, we demonstrate that convex, and in particular semidefinite, optimization methods lead to interesting and often unexpected answers to these questions.
Multiproduct systems with both setup times and costs: Fluid bounds and schedules
 Operations Research
, 2004
"... This paper considers a multiproduct, singleserver production system where both setup times and costs are incurred whenever the server changes product. The system is maketoorder with a per unit backlogging cost. The objective is to minimize the longrun average cost per unit time. Using a fluid m ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
This paper considers a multiproduct, singleserver production system where both setup times and costs are incurred whenever the server changes product. The system is maketoorder with a per unit backlogging cost. The objective is to minimize the longrun average cost per unit time. Using a fluid model, we provide a closedform lower bound on system performance. This bound is also shown to provide a lower bound for stochastic systems when scheduling is static, but is only an approximation when scheduling is dynamic. Heavytraffic analysis yields a refined bound that includes secondmoment terms. The fluid bound suggests both dynamic and static scheduling In this paper we consider a production environment where a number of different products are produced on a single machine and setup activities are necessary when switches of product type are made. These setup activities require both time and cost that depend on the specific product type. Throughout the paper we assume that the setups do not depend on the previous product produced
Linear programming performance bounds for Markov chains with polyhedrally translation invariant transition probabilities and applications to unreliable manufacturing systems and enhanced . . .
, 2001
"... Our focus is on a class of Markov chains which have a polyhedral translation invariance property for the transition probabilities. This class can be used to model several applications of interest which feature complexities not found in usual models of queueing networks, for example failure prone man ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Our focus is on a class of Markov chains which have a polyhedral translation invariance property for the transition probabilities. This class can be used to model several applications of interest which feature complexities not found in usual models of queueing networks, for example failure prone manufacturing systems which are operating under hedging point policies, or enhanced wafer fab models featuring batch tools and setups or affine index policies. We present a new family of performance bounds which is more powerful both in expressive capability as well as the quality of the bounds than some earlier approaches.
Exponential penalty function control with queues
 Annals of Applied Probability
, 2002
"... We introduce penaltyfunctionbased admission control policies to approximately maximize the expected reward rate in a loss network. These control policies are easy to implement and perform well both in the transient period as well as in steady state. A major advantage of the penalty approach is tha ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
We introduce penaltyfunctionbased admission control policies to approximately maximize the expected reward rate in a loss network. These control policies are easy to implement and perform well both in the transient period as well as in steady state. A major advantage of the penalty approach is that it avoids solving the associated dynamic program. However, a disadvantage of this approach is that it requires the capacity requested by individual requests to be sufficiently small compared to total available capacity. We first solve a related deterministic linear program (LP) and then translate an optimal solution of the LP into an admission control policy for the loss network via an exponential penalty function. We show that the penalty policy is a targettracking policy—it performs well because the optimal solution of the LP is a good target. We demonstrate that the penalty approach can be extended to track arbitrarily defined target sets. Results from preliminary simulation studies are included. 1. Introduction. We
Computing Stationary Probability Distributions and Large Deviation Rates for Constrained Random Walks. The Undecidability Results
, 2002
"... Our model is a constrained homogeneous random walk in + . The convergence to stationarity for such a random walk can often be checked by constructing a Lyapunov function. The same Lyapunov function can also be used for computing approximately the stationary distribution of this random walk, us ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Our model is a constrained homogeneous random walk in + . The convergence to stationarity for such a random walk can often be checked by constructing a Lyapunov function. The same Lyapunov function can also be used for computing approximately the stationary distribution of this random walk, using methods developed by Meyn and Tweedie in [34]. In this paper we show that, for stationary homogeneous random walks, computing the stationary probability exactly is an undecidable problem, even if a Lyapunov function is available. That is no algorithm can exist to achieve this task. We then prove that computing large deviation rates for this model is also an undecidable problem. We extend these results to a certain type of queueing systems. The implication of these results is that no useful formulas for computing stationary probabilities and large deviations rates can exist in these systems.
Performance Analysis of Queueing Networks via Robust Optimization
, 2010
"... Performance analysis of queueing networks is one of the most challenging areas of queueing theory. Barring very specialized models such as productform type queueing networks, there exist very few results which provide provable nonasymptotic upper and lower bounds on key performance measures. In th ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Performance analysis of queueing networks is one of the most challenging areas of queueing theory. Barring very specialized models such as productform type queueing networks, there exist very few results which provide provable nonasymptotic upper and lower bounds on key performance measures. In this paper we propose a new performance analysis method, which is based on the robust optimization. The basic premise of our approach is as follows: rather than assuming that the stochastic primitives of a queueing model satisfy certain probability laws, such as, for example, i.i.d. interarrival and service times distributions, we assume that the underlying primitives are deterministic and satisfy the implications of such probability laws. These implications take the form of simple linear constraints, namely, those motivated by the Law of the Iterated Logarithm (LIL). Using this approach we are able to obtain performance bounds on some key performance measures. Furthermore, these performance bounds imply similar bounds in the underlying stochastic queueing models.