Results 1  10
of
62
The Markov Chain Monte Carlo method: an approach to approximate counting and integration
, 1996
"... In the area of statistical physics, Monte Carlo algorithms based on Markov chain simulation have been in use for many years. The validity of these algorithms depends crucially on the rate of convergence to equilibrium of the Markov chain being simulated. Unfortunately, the classical theory of stocha ..."
Abstract

Cited by 234 (13 self)
 Add to MetaCart
In the area of statistical physics, Monte Carlo algorithms based on Markov chain simulation have been in use for many years. The validity of these algorithms depends crucially on the rate of convergence to equilibrium of the Markov chain being simulated. Unfortunately, the classical theory of stochastic processes hardly touches on the sort of nonasymptotic analysis required in this application. As a consequence, it had previously not been possible to make useful, mathematically rigorous statements about the quality of the estimates obtained. Within the last ten years, analytical tools have been devised with the aim of correcting this deficiency. As well as permitting the analysis of Monte Carlo algorithms for classical problems in statistical physics, the introduction of these tools has spurred the development of new approximation algorithms for a wider class of problems in combinatorial enumeration and optimization. The “Markov chain Monte Carlo ” method has been applied to a variety of such problems, and often provides the only known efficient (i.e., polynomial time) solution technique.
An Introduction to MCMC for Machine Learning
, 2003
"... This purpose of this introductory paper is threefold. First, it introduces the Monte Carlo method with emphasis on probabilistic machine learning. Second, it reviews the main building blocks of modern Markov chain Monte Carlo simulation, thereby providing and introduction to the remaining papers of ..."
Abstract

Cited by 222 (2 self)
 Add to MetaCart
This purpose of this introductory paper is threefold. First, it introduces the Monte Carlo method with emphasis on probabilistic machine learning. Second, it reviews the main building blocks of modern Markov chain Monte Carlo simulation, thereby providing and introduction to the remaining papers of this special issue. Lastly, it discusses new interesting research horizons.
Random Walks And An O*(n 5 ) Volume Algorithm For Convex Bodies
, 1996
"... Given a high dimensional convex body K ` IR n by a separation oracle, we can approximate its volume with relative error ", using O (n 5 ) oracle calls. Our algorithm also brings the body into isotropic position. As all previous randomized volume algorithms, we use "rounding" followed by a mul ..."
Abstract

Cited by 75 (8 self)
 Add to MetaCart
Given a high dimensional convex body K ` IR n by a separation oracle, we can approximate its volume with relative error ", using O (n 5 ) oracle calls. Our algorithm also brings the body into isotropic position. As all previous randomized volume algorithms, we use "rounding" followed by a multiphase MonteCarlo (product estimator) technique. Both parts rely on sampling (generating random points in K), which is done by random walk. Our algorithm introduces three new ideas: ffl the use of the isotropic position (or at least an approximation of it) for rounding, ffl the separation of global obstructions (diameter) and local obstructions (boundary problems) for fast mixing, and ffl a stepwise interlacing of rounding and sampling. 1 . Introduction For a variety of geometric objects, classical results characterize various geometric parameters. Many of these results are useful even in practical situations: they can easily be transformed into efficient algorithms. Some other theorem...
Isoperimetric Problems for Convex Bodies and a Localization Lemma
, 1995
"... We study the smallest number /(K) such that a given convex body K in IR n can be cut into two parts K 1 and K 2 by a surface with an (n \Gamma 1)dimensional measure /(K)vol(K 1 ) \Delta vol(K 2 )=vol(K). Let M 1 (K) be the average distance of a point of K from its center of gravity. We prove for ..."
Abstract

Cited by 73 (8 self)
 Add to MetaCart
We study the smallest number /(K) such that a given convex body K in IR n can be cut into two parts K 1 and K 2 by a surface with an (n \Gamma 1)dimensional measure /(K)vol(K 1 ) \Delta vol(K 2 )=vol(K). Let M 1 (K) be the average distance of a point of K from its center of gravity. We prove for the "isoperimetric coefficient" that /(K) ln 2 M 1 (K) ; and give other upper and lower bounds. We conjecture that our upper bound is best possible up to a constant. Our main tool is a general "Localization Lemma" that reduces integral inequalities over the ndimensional space to integral inequalities in a single variable. This lemma was first proved by two of the authors in an earlier paper, but here we give various extensions and variants that make its application smoother. We illustrate the usefulness of the lemma by showing how a number of wellknown results can be proved using it.
Computing The Volume Of Convex Bodies: A Case Where Randomness Provably Helps
, 1991
"... We discuss the problem of computing the volume of a convex body K in IR n . We review worstcase results which show that it is hard to deterministically approximate volnK and randomised approximation algorithms which show that with randomisation one can approximate very nicely. We then provide som ..."
Abstract

Cited by 62 (6 self)
 Add to MetaCart
We discuss the problem of computing the volume of a convex body K in IR n . We review worstcase results which show that it is hard to deterministically approximate volnK and randomised approximation algorithms which show that with randomisation one can approximate very nicely. We then provide some applications of this latter result. Supported by NATO grant RG0088/89 y Supported by NSF grant CCR8900112 and NATO grant RG0088/89 1 Introduction The mathematical study of areas and volumes is as old as civilization itself, and has been conducted for both intellectual and practical reasons. As far back as 2000 B.C., the Egyptians 1 had methods for approximating the areas of fields (for taxation purposes) and the volumes of granaries. The exact study of areas and volumes began with Euclid 2 and was carried to a high art form by Archimedes 3 . The modern study of this subject began with the great astronomer Johann Kepler's treatise 4 Nova stereometria doliorum vinariorum, wh...
Learning an Agent's Utility Function by Observing Behavior
 In Proc. of the 18th Int’l Conf. on Machine Learning
, 2001
"... This paper considers the task of predicting the future decisions of an agent A based on his past decisions. We assume that A is rational  he uses the principle of maximum expected utility. We also assume that the probability distribution P he assigns to random events is known, so that we need only ..."
Abstract

Cited by 54 (0 self)
 Add to MetaCart
This paper considers the task of predicting the future decisions of an agent A based on his past decisions. We assume that A is rational  he uses the principle of maximum expected utility. We also assume that the probability distribution P he assigns to random events is known, so that we need only infer his utility function u to model his decision process. We consider the task of using A's previous decisions to learn about u. In particular, A's past decisions can be viewed as constraints on u. If we have a prior probability distribution p(u) over u (e.g., learned from a set of utility functions in the population), we can then condition on these constraints to obtain a posterior distribution q(u). We present an efficient Markov Chain Monte Carlo scheme to generate samples from q(u), which can be used to estimate not only a single "expected" course of action for A, but a distribution over possible courses of action. We show that this capability is particularly useful in a twoplayer setting where a second learning agent is trying to optimize her own payoff, which also depends on A's actions and utilities.
Bayesian inverse reinforcement learning
 in 20th Int. Joint Conf. Artificial Intelligence
, 2007
"... Inverse Reinforcement Learning (IRL) is the problem of learning the reward function underlying a Markov Decision Process given the dynamics of the system and the behaviour of an expert. IRL is motivated by situations where knowledge of the rewards is a goal by itself (as in preference elicitation) a ..."
Abstract

Cited by 53 (0 self)
 Add to MetaCart
Inverse Reinforcement Learning (IRL) is the problem of learning the reward function underlying a Markov Decision Process given the dynamics of the system and the behaviour of an expert. IRL is motivated by situations where knowledge of the rewards is a goal by itself (as in preference elicitation) and by the task of apprenticeship learning (learning policies from an expert). In this paper we show how to combine prior knowledge and evidence from the expert’s actions to derive a probability distribution over the space of reward functions. We present efficient algorithms that find solutions for the reward learning and apprenticeship learning tasks that generalize well over these distributions. Experimental results show strong improvement for our methods over previous heuristicbased approaches. 1
Markov Chains and Polynomial time Algorithms
, 1994
"... This paper outlines the use of rapidly mixing Markov Chains in randomized polynomial time algorithms to solve approximately certain counting problems. They fall into two classes: combinatorial problems like counting the number of perfect matchings in certain graphs and geometric ones like computing ..."
Abstract

Cited by 48 (0 self)
 Add to MetaCart
This paper outlines the use of rapidly mixing Markov Chains in randomized polynomial time algorithms to solve approximately certain counting problems. They fall into two classes: combinatorial problems like counting the number of perfect matchings in certain graphs and geometric ones like computing the volumes of convex sets.
Adiabatic quantum state generation and statistical zeroknowledge
 in Proc. 35th STOC
, 2003
"... The design of new quantum algorithms has proven to be an extremely difficult task. This paper considers a different approach to the problem. We systematically study ’quantum state generation’, namely, which superpositions can be efficiently generated. We first show that all problems in Statistical Z ..."
Abstract

Cited by 43 (3 self)
 Add to MetaCart
The design of new quantum algorithms has proven to be an extremely difficult task. This paper considers a different approach to the problem. We systematically study ’quantum state generation’, namely, which superpositions can be efficiently generated. We first show that all problems in Statistical Zero Knowledge (SZK), a class which contains many languages that are natural candidates for BQP, can be reduced to an instance of quantum state generation. This was known before for graph isomorphism, but we give a general recipe for all problems in SZK. We demonstrate the reduction from the problem to its quantum state generation version for three examples: Discrete log, quadratic residuosity and a gap version of closest vector in a lattice. We then develop tools for quantum state generation. For this task, we define the framework of ’adiabatic quantum state generation ’ which uses the language of ground states, spectral gaps and Hamiltonians instead of the standard unitary gate language. This language stems from the recently suggested adiabatic computation model [20] and seems to be especially tailored for the task of quantum state generation. After defining the paradigm, we provide two basic lemmas for adiabatic quantum state generation: • The Sparse Hamiltonian lemma, which gives a general technique for implementing sparse Hamiltonians efficiently, and, • The jagged adiabatic path lemma, which gives conditions for a sequence of Hamiltonians to allow efficient adiabatic state generation. We use our tools to prove that any quantum state which can be generated efficiently in the standard model can also be generated efficiently adiabatically, and vice versa. Finally we show how to apply our techniques to generate superpositions corresponding to limiting distributions of a large class of Markov chains, including the uniform distribution over all perfect