Results 1  10
of
10
A tail inequality for suprema of unbounded empirical processes with applications to Markov chains
, 2008
"... ..."
Concentration inequalities for dependent random variables via the martingale method
 ANNALS OF PROBABILITY
, 2008
"... The martingale method is used to establish concentration inequalities for a class of dependent random sequences on a countable state space, with the constants in the inequalities expressed in terms of certain mixing coefficients. Along the way, bounds are obtained on martingale differences associate ..."
Abstract

Cited by 19 (4 self)
 Add to MetaCart
The martingale method is used to establish concentration inequalities for a class of dependent random sequences on a countable state space, with the constants in the inequalities expressed in terms of certain mixing coefficients. Along the way, bounds are obtained on martingale differences associated with the random sequences, which may be of independent interest. As applications of the main result, concentration inequalities are also derived for inhomogeneous Markov chains and hidden Markov chains, and an extremal property associated with their martingale difference bounds is established. This work complements and generalizes certain concentration inequalities obtained by Marton and Samson, while also providing different proofs of some known results.
Simulationbased uniform value function estimates of discounted and averagereward MDPs
 SIAM Journal on Control and Optimization
, 2004
"... Abstract — The value function of a Markov decision problem assigns to each policy its expected discounted reward. This expected reward can be estimated as the empirical average of the reward over many independent simulation runs. We derive bounds on the number of runs needed for the convergence of t ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
Abstract — The value function of a Markov decision problem assigns to each policy its expected discounted reward. This expected reward can be estimated as the empirical average of the reward over many independent simulation runs. We derive bounds on the number of runs needed for the convergence of the empirical average to the expected reward uniformly for a class of policies, in terms of the VC or pseudo dimension of the policy class. Uniform convergence results are also obtained for the average reward case. They can be extended to partially observed MDPs and Markov games. The results can be viewed as an extension of the probably approximately correct (PAC) learning theory for partially observable MDPs (POMDPs) and Markov games. I.
Large deviation asymptotics and control variates for simulating large functions,” 2004, submitted for publication
"... Consider the normalized partial sums of a realvalued function F of a Markov chain, φn: = n −1 n−1 F(Φ(k)), n ≥ 1. k=0 The chain {Φ(k) : k ≥ 0} takes values in a general state space X, with transition kernel P, and it is assumed that the Lyapunov drift condition holds: PV ≤ V −W +bIC where V: X → (0 ..."
Abstract

Cited by 6 (5 self)
 Add to MetaCart
Consider the normalized partial sums of a realvalued function F of a Markov chain, φn: = n −1 n−1 F(Φ(k)), n ≥ 1. k=0 The chain {Φ(k) : k ≥ 0} takes values in a general state space X, with transition kernel P, and it is assumed that the Lyapunov drift condition holds: PV ≤ V −W +bIC where V: X → (0, ∞), W: X → [1, ∞), the set C is small, and W dominates F. Under these assumptions, the following conclusions are obtained: (i) It is known that this drift condition is equivalent to the existence of a unique invariant distribution π satisfying π(W) < ∞, and the Law of Large Numbers holds for any function F dominated by W: φn → φ: = π(F), a.s., n → ∞. (ii) The lower error probability defined by P{φn ≤ c}, for c < φ, n ≥ 1, satisfies a large deviation limit theorem when the function F satisfies a monotonicity condition. Under additional minor conditions an exact large deviations expansion is obtained. (iii) If W is nearmonotone then controlvariates are constructed based on the Lyapunov function V, providing a pair of estimators that together satisfy nontrivial large asymptotics for the lower and upper error probabilities. In an application to simulation of queues it is shown that exact large deviation asymptotics are possible even when the estimator does not satisfy a Central Limit Theorem.
Rigorous confidence bounds for MCMC under a geometric drift condition
, 2009
"... Abstract: We assume a drift condition towards a small set and bound the mean square error of estimators obtained by taking averages along a single trajectory of a Markov chain Monte Carlo algorithm. We use these bounds to construct fixedwidth nonasymptotic confidence intervals. For a possibly unbou ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
Abstract: We assume a drift condition towards a small set and bound the mean square error of estimators obtained by taking averages along a single trajectory of a Markov chain Monte Carlo algorithm. We use these bounds to construct fixedwidth nonasymptotic confidence intervals. For a possibly unbounded function f: X → R, let I = ∫ f(x)π(x)dx be the value of X interest and Ît,n = (1/n) ∑ t+n−1 f(Xi) its MCMC estimate. Precisely, i=t we derive lower bounds for the length of the trajectory n and burnin time t which ensure that P(  Ît,n − I  ≤ ε) ≥ 1 − α. The bounds depend only and explicitly on drift parameters, on the V −norm of f, where V is the drift function and on precision and confidence parameters ε, α. Next we analyse an MCMC estimator based on the median of multiple shorter runs that allows for sharper bounds for the required total simulation cost. In particular the methodology can be applied for computing Bayesian estimators in practically relevant models. We illustrate our bounds numerically in a simple example.
Online Pairing of VoIP Conversations
"... This paper answers the following question; given a multiplicity of evolving 1way conversations, can a machine or an algorithm discern the conversational pairs in an online fashion, without understanding the content of the communications? Our analysis indicates that this is possible, and can be achi ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
This paper answers the following question; given a multiplicity of evolving 1way conversations, can a machine or an algorithm discern the conversational pairs in an online fashion, without understanding the content of the communications? Our analysis indicates that this is possible, and can be achieved just by exploiting the temporal dynamics inherent in a conversation. We also show that our findings are applicable for anonymous and encrypted conversations over VoIP networks. We achieve this by exploiting the aperiodic interdeparture time of VoIP packets, hence trivializing each VoIP stream into a binary timeseries, indicating the voice activity of each stream. We propose effective techniques that progressively pair conversing parties with high accuracy and in a limited amount of time. Our findings are verified empirically on a dataset consisting of 1000 conversations. We obtain very high pairing accuracy that reaches 97 % after 5 minutes of voice conversations. Using a modeling approach we also demonstrate analytically that our result can be extended over an unlimited number of conversations.
unknown title
, 2005
"... www.elsevier.com/locate/spa Worstcase largedeviation asymptotics with application to queueing and information theory ✩ Charuhas Pandit a, Sean Meyn b,∗ ..."
Abstract
 Add to MetaCart
www.elsevier.com/locate/spa Worstcase largedeviation asymptotics with application to queueing and information theory ✩ Charuhas Pandit a, Sean Meyn b,∗