DMCA
Information-theoretic regret bounds for Gaussian process optimization in the bandit setting. Information Theory (2012)
| Venue: | IEEE Transactions on |
| Citations: | 26 - 3 self |
BibTeX
@ARTICLE{Srinivas12information-theoreticregret,
author = {Niranjan Srinivas and Andreas Krause and Sham M. Kakade and Matthias W. Seeger},
title = {Information-theoretic regret bounds for Gaussian process optimization in the bandit setting. Information Theory},
journal = {IEEE Transactions on},
year = {2012},
pages = {3265}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract—Many applications require optimizing an unknown, noisy function that is expensive to evaluate. We formalize this task as a multiarmed bandit problem, where the payoff function is either sampled from a Gaussian process (GP) or has low norm in a reproducing kernel Hilbert space. We resolve the important open problem of deriving regret bounds for this setting, which imply novel convergence rates for GP optimization. We analyze an intuitive Gaussian process upper confidence bound (- algorithm, and bound its cumulative regret in terms of maximal information gain, establishing a novel connection between GP optimization and experimental design. Moreover, by bounding the latter in terms of operator spectra, we obtain explicit sublinear regret bounds for many commonly used covariance functions. In some important cases, our bounds have surprisingly weak dependence on the dimensionality. In our experiments on real sensor data,- compares favorably with other heuristical GP optimization approaches. Index Terms—Bandit problems, Bayesian prediction, experimental design, Gaussian process (GP), information gain,
Keyphrases
gaussian process optimization information-theoretic regret bound information theory gp optimization gaussian process experimental design low norm cumulative regret abstract many application payoff function covariance function important case novel convergence rate bayesian prediction important open problem explicit sublinear regret bound novel connection regret bound real sensor data index term bandit problem noisy function maximal information gain operator spectrum weak dependence heuristical gp optimization approach reproducing kernel hilbert space multiarmed bandit problem information gain
