In this paper we derive conditions for geometric ergodicity of the random walk-based Metropolis algorithm on R k . We show that at least exponentially light tails of the target density is a necessity. This extends the one-dimensional result of (Mengersen and Tweedie, 1996). For sub-exponential target densities we characterize the geometrically ergodic algorithms and we derive a practical sufficient condition which is stable under addition and multiplication. This condition is especially satisfied for the class of densities considered in (Roberts and Tweedie, 1996).

this paper, we analyse theoretical properties of the slice sampler. We find that the algorithm has extremely robust geometric ergodicity properties. For the case of just one auxiliary variable, we demonstrate that the algorithm is stochastic monotone, and deduce analytic bounds on the total variation distance from stationarity of the method using Foster-Lyapunov drift condition methodology. 1. Introduction.

In this paper we study the ergodicity properties of some adaptive Monte Carlo Markov chain algorithms (MCMC) that have been recently proposed in the literature. We prove that under a set of verifiable conditions, ergodic averages calculated from the output of a so-called adaptive MCMC sampler converge to the required value and can even, under more stringent assumptions, satisfy a central limit theorem. We prove that the conditions required are satisfied for the Independent Metropolis-Hastings algorithm and the Random Walk Metropolis algorithm with symmetric increments. Finally we propose an application of these results to the case where the proposal distribution of the Metropolis-Hastings update is a mixture of distributions from a curved exponential family.

In this paper we consider Foster-Lyapunov type drift conditions for Markov chains which imply polynomial rate convergence to stationarity in appropriate V -norms. We also show how these results can be used to prove Central Limit Theorems for functions of the Markov chain. Examples are considered to random walks on the half line and the independence sampler.

The goal of this mainly expository paper is to describe conditions which guarantee a central limit theorem for functionals of general state space Markov chains with a view towards Markov chain Monte Carlo settings. Thus the focus is on the connections between drift and mixing conditions and their implications. In particular, we consider three commonly cited central limit theorems and discuss their relationship to classical results for mixing processes. Several motivating examples are given which range from toy one-dimensional settings to complicated settings encountered in Markov chain Monte Carlo. 1

In principle, a wide variety of sequential decision problems -- ranging from dynamic resource allocation in telecommunication networks to financial risk management -- can be formulated in terms of stochastic control and solved by the algorithms of dynamic programming. Such algorithms compute and store a value function, which evaluates expected future reward as a function of current state. Unfortunately, exact computation of the value function typically requires time and storage that grow proportionately with the number of states, and consequently, the enormous state spaces that arise in practical applications render the algorithms intractable. In this thesis, we study tractable methods that approximate the value function. Our work builds on research in an area of artificial intelligence known as reinforcement learning. A point of focus of this thesis is temporal-difference learning -- a stochastic algorithm inspired to some extent by phenomena observed in animal behavior. Given a selection of...

Recently, a novel concept for the computation of essential features of the dynamics of Hamiltonian systems (such as molecular dynamics) has been proposed [1]. The realization of this concept had been based on subdivision techniques applied to the Frobenius-Perron operator for the dynamical system. The present paper suggests an alternative but related concept that merges the conceptual advantages of the dynamical systems approach with the appropriate statistical physics framework. This approach allows to de ne the phrase "conformation" in terms of the dynamical behavior of the molecular system and to characterize the dynamical stability of conformations. In a first step, the frequency of conformational changes is characterized in statistical terms leading to the definition of some Markov operator T that describes the corresponding transition probabilities within the canonical ensemble. In a second step, a discretization of T via specific hybrid Monte Carlo techniques is shown ...

In spite of many applications of evolutionary algorithms in optimisation, theoretical results on the computation time and time complexity of evolutionary algorithms on different optimisation problems are relatively few. It is still unclear when an evolutionary algorithm is expected to solve an optimisation problem efficiently or otherwise. This paper gives a general analytic framework for analysing first hitting times of evolutionary algorithms. The framework is built on the absorbing Markov chain model of evolutionary algorithms. The first step towards a systematic comparative study among different EAs and their first hitting times has been made in the paper.

In this paper, we establish a sufficient condition for the stability of a multiclass fluid network and queueing network under priority service disciplines. The sufficient condition is based on the existence of a linear Lyapunov function, and it is stated in terms of the feasibility of a set of inequalities that are defined by network parameters. In all the networks we have tested, this sufficient condition actually gives a necessary and sufficient condition for their stability.

Consider the partial sums {St} of a real-valued functional F(�(t)) of a Markov chain {�(t)} with values in a general state space. Assuming only that the Markov chain is geometrically ergodic and that the functional F is bounded, the following conclusions are obtained: Spectral theory. Well-behaved solutions fˇ can be constructed for the “multiplicative Poisson equation ” (eαF P) f ˇ = λf ˇ,wherePis the transition kernel of the Markov chain and α ∈ C is a constant. The function fˇ is an eigenfunction, with corresponding eigenvalue λ, for the kernel (eαF P) = eαF(x) P(x,dy). A “multiplicative ” mean ergodic theorem. For all complex α in a neighborhood of the origin, the normalized mean of exp(αSt) (and not the logarithm of the mean) converges to fˇ exponentially fast, where fˇ is a solution of the multiplicative Poisson equation. Edgeworth expansions. Rates are obtained for the convergence of the distribution function of the normalized partial sums St to the standard Gaussian distribution. The first term in this expansion is of order (1 / √ t) and it depends on the initial condition of the Markov chain through the solution ̂ F of the associated Poisson equation (and not the solution fˇ of the multiplicative Poisson equation). Large deviations. The partial sums are shown to satisfy a large deviations principle in a neighborhood of the mean. This result, proved under geometric ergodicity alone, cannot in general be extended to the whole real line. Exact large deviations asymptotics. Rates of convergence are obtained for the large deviations estimates above. The polynomial preexponent is of order (1 / √ t) and its coefficient depends on the initial condition of the Markov chain through the solution fˇ of the multiplicative Poisson equation. Extensions of these results to continuous-time Markov processes are also given. 1. Introduction. Consider a Markov process � ={�(t): t ∈ T