Results 11  20
of
439
General state space Markov chains and MCMC algorithm
 PROBABILITY SURVEYS
, 2004
"... This paper surveys various results about Markov chains on general (noncountable) state spaces. It begins with an introduction to Markov chain Monte Carlo (MCMC) algorithms, which provide the motivation and context for the theory which follows. Then, sufficient conditions for geometric and uniform e ..."
Abstract

Cited by 106 (26 self)
 Add to MetaCart
This paper surveys various results about Markov chains on general (noncountable) state spaces. It begins with an introduction to Markov chain Monte Carlo (MCMC) algorithms, which provide the motivation and context for the theory which follows. Then, sufficient conditions for geometric and uniform ergodicity are presented, along with quantitative bounds on the rate of convergence to stationarity. Many of these results are proved using direct coupling constructions based on minorisation and drift conditions. Necessary and sufficient conditions for Central Limit Theorems (CLTs) are also presented, in some cases proved via the Poisson Equation or direct regeneration constructions. Finally, optimal scaling and weak convergence results for MetropolisHastings algorithms are discussed. None of the results presented is new, though many of the proofs are. We also describe some Open Problems.
Crosslayer congestion control, routing and scheduling design in ad hoc wireless networks
 Proc. IEEE Infocom
, 2006
"... Abstract — This paper considers jointly optimal design of crosslayer congestion control, routing and scheduling for ad hoc wireless networks. We first formulate the rate constraint and scheduling constraint using multicommodity flow variables, and formulate resource allocation in networks with fixed ..."
Abstract

Cited by 97 (11 self)
 Add to MetaCart
Abstract — This paper considers jointly optimal design of crosslayer congestion control, routing and scheduling for ad hoc wireless networks. We first formulate the rate constraint and scheduling constraint using multicommodity flow variables, and formulate resource allocation in networks with fixed wireless channels (or singlerate wireless devices that can mask channel variations) as a utility maximization problem with these constraints. By dual decomposition, the resource allocation problem naturally decomposes into three subproblems: congestion control, routing and scheduling that interact through congestion price. The global convergence property of this algorithm is proved. We next extend the dual algorithm to handle networks with timevarying channels and adaptive multirate devices. The stability of the resulting system is established, and its performance is characterized with respect to an ideal reference system which has the best feasible rate region at link layer. We then generalize the aforementioned results to a general model of queueing network served by a set of interdependent parallel servers with timevarying service capabilities, which models many design problems in communication networks. We show that for a general convex optimization problem where a subset of variables lie in a polytope and the rest in a convex set, the dualbased algorithm remains stable and optimal when the constraint set is modulated by an irreducible finitestate Markov chain. This paper thus presents a step toward a systematic way to carry out crosslayer design in the framework of “layering as optimization decomposition ” for timevarying channel models. I.
Geometric Ergodicity and Hybrid Markov Chains
, 1997
"... Various notions of geometric ergodicity for Markov chains on general state spaces exist. In this paper, we review certain relations and implications among them. We then apply these results to a collection of chains commonly used in Markov chain Monte Carlo simulation algorithms, the socalled hybrid ..."
Abstract

Cited by 75 (24 self)
 Add to MetaCart
Various notions of geometric ergodicity for Markov chains on general state spaces exist. In this paper, we review certain relations and implications among them. We then apply these results to a collection of chains commonly used in Markov chain Monte Carlo simulation algorithms, the socalled hybrid chains. We prove that under certain conditions, a hybrid chain will "inherit" the geometric ergodicity of its constituent parts. 1 Introduction A question of increasing importance in the Markov chain Monte Carlo literature (Gelfand and Smith, 1990; Smith and Roberts, 1993) is the issue of geometric ergodicity of Markov chains (Tierney, 1994, Section 3.2; Meyn and Tweedie, 1993, Chapters 15 and 16; Roberts and Tweedie, 1996). However, there are a number of different notions of the phrase "geometrically ergodic", depending on perspective (total variation distance vs. in L 2 ; with reference to a particular V function; etc.). One goal of this paper is to review and clarify the relationship...
Asymptotic results for multiplexing subexponential onoff processes
 Advances in Applied Probability
, 1998
"... Consider an aggregate arrival process AN obtained by multiplexing N OnOff processes with exponential Off periods of rate λ and subexponential On periods τon. As N goes to infinity, with λN → Λ, AN approaches an M/G/ ∞ type process. Both for finite and infinite N, we obtain the asymptotic characteri ..."
Abstract

Cited by 70 (20 self)
 Add to MetaCart
Consider an aggregate arrival process AN obtained by multiplexing N OnOff processes with exponential Off periods of rate λ and subexponential On periods τon. As N goes to infinity, with λN → Λ, AN approaches an M/G/ ∞ type process. Both for finite and infinite N, we obtain the asymptotic characterization of the arrival process activity period. Using these results we investigate a fluid queue with the limiting M/G/ ∞ arrival process A ∞ t and capacity c. When On periods are regularly varying (with noninteger exponent), we derive a precise asymptotic behavior of the queue length random variable QP t observed at the beginning of the arrival process activity periods P[Q P t +ρ−c> x] ∼ Λr P[τ c−ρ x/(r+ρ−c) on> u]du x → ∞, where ρ = EA ∞ t < c; r (c ≤ r) is the rate at which the fluid is arriving during an On period. The asymptotic (time average) queuedistributionlower boundis obtained undermoregeneral assumptions on On periods than regular variation. In addition, we analyze a queueing system in which one OnOff process, whose On period belongs to a subclass of subexponential distributions, is multiplexed with independent exponential processes with aggregate expected rate Eet. This system is shown to be asymptotically equivalent to the same queueing system with the exponential arrival processes being replaced by their total mean value Eet.
Power Allocation and Routing in Multibeam Satellites With TimeVarying Channels
 IEEE TRANSACTIONS ON NETWORKING
, 2003
"... We consider power and server allocation in a multibeam satellite downlink which transmits data to different ground locations over timevarying channels. Packets destined for each ground location are stored in separate queues and the server rate for each queue depends on the power ( ) allocated to th ..."
Abstract

Cited by 67 (16 self)
 Add to MetaCart
We consider power and server allocation in a multibeam satellite downlink which transmits data to different ground locations over timevarying channels. Packets destined for each ground location are stored in separate queues and the server rate for each queue depends on the power ( ) allocated to that server and the channel state ( ) according to a concave ratepower curve ( ). We establish the capacity region of all arrival rate vectors ( 1 ... ) which admit a stabilizable system. We then develop a powerallocation policy which stabilizes the system whenever the rate vector lies within the capacity region. Such stability is guaranteed even if the channel model and the specific arrival rates are unknown. Furthermore, the algorithm is shown to be robust to arbitrary variations in the input rates and a bound on average delay is established. As a special case, this analysis verifies stability and provides a performance bound for the Choosethe LargestConnectedQueues policy when channels can be in one of two states (ON or OFF) and servers are allocated at every timestep ( ). These results are extended to treat a joint problem of routing and power allocation in a system with multiple users and satellites and a throughput maximizing algorithm for this joint problem is constructed. Finally, we address the issue of interchannel interference and develop a modified policy when power vectors are constrained to feasible activation sets. Our analysis and problem formulation is also applicable to power control for wireless systems.
Convergence rates of Markov chains
, 1995
"... this paper, we attempt to describe various mathematical techniques which have been used to bound such rates of convergence. In particular, we describe eigenvalue analysis, random walks on groups, coupling, and minorization conditions. Connections are made to modern areas of research wherever possibl ..."
Abstract

Cited by 62 (4 self)
 Add to MetaCart
this paper, we attempt to describe various mathematical techniques which have been used to bound such rates of convergence. In particular, we describe eigenvalue analysis, random walks on groups, coupling, and minorization conditions. Connections are made to modern areas of research wherever possible. Elements of linear algebra, probability theory, group theory, and measure theory are used, but efforts are made to keep the presentation elementary and accessible. Acknowledgements. I thank Eric Belsley for comments and corrections, and thank Persi Diaconis for introducing me to this subject and teaching me so much. 1. Introduction and motivation.
A survey of maxtype recursive distributional equations
 Annals of Applied Probability 15 (2005
, 2005
"... In certain problems in a variety of applied probability settings (from probabilistic analysis of algorithms to statistical physics), the central requirement is to solve a recursive distributional equation of the form X d = g((ξi,Xi), i ≥ 1). Here(ξi) and g(·) are given and the Xi are independent cop ..."
Abstract

Cited by 62 (6 self)
 Add to MetaCart
In certain problems in a variety of applied probability settings (from probabilistic analysis of algorithms to statistical physics), the central requirement is to solve a recursive distributional equation of the form X d = g((ξi,Xi), i ≥ 1). Here(ξi) and g(·) are given and the Xi are independent copies of the unknown distribution X. We survey this area, emphasizing examples where the function g(·) is essentially a “maximum ” or “minimum” function. We draw attention to the theoretical question of endogeny: inthe associated recursive tree process X i,aretheX i measurable functions of the innovations process (ξ i)? 1. Introduction. Write
Effective bandwidths: call admission, traffic policing, and filtering for ATM networks
, 1994
"... In this paper we review and extend the effective bandwidth results of Kelly [28], and Kesidis, Walrand and Chang [29, 6]. These results provide a framework for call admission schemes which are sensitive to constraints on the mean delay or the tail distribution of the workload in buffered queues. We ..."
Abstract

Cited by 60 (9 self)
 Add to MetaCart
In this paper we review and extend the effective bandwidth results of Kelly [28], and Kesidis, Walrand and Chang [29, 6]. These results provide a framework for call admission schemes which are sensitive to constraints on the mean delay or the tail distribution of the workload in buffered queues. We present results which are valid for a wide variety of traffic streams and discuss their applicability for traffic management in ATM networks. We discuss the impact of traffic policing schemes, such as thresholding and ltering, on the effective bandwidth of sources. Finally we discuss effective bandwidth results for Brownian traffic models for which explicit results reveal the interaction arising in finite buffers.
The CrossEntropy Method for Combinatorial and Continuous Optimization
, 1999
"... We present a new and fast method, called the crossentropy method, for finding the optimal solution of combinatorial and continuous nonconvex optimization problems with convex bounded domains. To find the optimal solution we solve a sequence of simple auxiliary smooth optimization problems based on ..."
Abstract

Cited by 58 (7 self)
 Add to MetaCart
We present a new and fast method, called the crossentropy method, for finding the optimal solution of combinatorial and continuous nonconvex optimization problems with convex bounded domains. To find the optimal solution we solve a sequence of simple auxiliary smooth optimization problems based on KullbackLeibler crossentropy, importance sampling, Markov chain and Boltzmann distribution. We use importance sampling as an important ingredient for adaptive adjustment of the temperature in the Boltzmann distribution and use KullbackLeibler crossentropy to find the optimal solution. In fact, we use the mode of a unimodal importance sampling distribution, like the mode of beta distribution, as an estimate of the optimal solution for continuous optimization and Markov chains approach for combinatorial optimization. In the later case we show almost surely convergence of our algorithm to the optimal solution. Supporting numerical results for both continuous and combinatorial optimization problems are given as well. Our empirical studies suggest that the crossentropy method has polynomial in the size of the problem running time complexity.
Likelihood Ratio Gradient Estimation For Stochastic Recursions
 Communications of the ACM
, 1995
"... . In this paper, we develop mathematical machinery for verifying that a broad class of general state space Markov chains reacts smoothly to certain types of perturbations in the underlying transition structure. Our main result provides conditions under which the stationary probability measure of an ..."
Abstract

Cited by 55 (7 self)
 Add to MetaCart
. In this paper, we develop mathematical machinery for verifying that a broad class of general state space Markov chains reacts smoothly to certain types of perturbations in the underlying transition structure. Our main result provides conditions under which the stationary probability measure of an ergodic Harris recurrent Markov chain is differentiable in a certain strong sense. The approach is based on likelihood ratio "changeofmeasure" arguments, and leads directly to a "likelihood ratio gradient estimator" that can be computed numerically. Keywords: Harris recurrent Markov chain, likelihood ratio, gradient estimation, regeneration. 1 The research of this author was supported by the U. S. Army Research Office under Contract No. DAAL0391G 0101 and by the National Science Foundation under Contract No. DDM9101580. 2 This author's research was supported by NSERCCanada grant No. OGP0110050 and FCARQu'ebec grant No. 93ER1654. 1. Introduction In this paper, we will study the cl...