DMCA
Contextual Gaussian Process Bandit Optimization
| Citations: | 15 - 2 self |
BibTeX
@MISC{Krause_contextualgaussian,
author = {Andreas Krause and Cheng Soon Ong},
title = {Contextual Gaussian Process Bandit Optimization},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
How should we design experiments to maximize performance of a complex system, taking into account uncontrollable environmental conditions? How should we select relevant documents (ads) to display, given information about the user? These tasks can be formalized as contextual bandit problems, where at each round, we receive context (about the experimental conditions, the query), and have to choose an action (parameters, documents). The key challenge is to trade off exploration by gathering data for estimating the mean payoff function over the context-action space, and to exploit by choosing an action deemed optimal based on the gathered data. We model the payoff function as a sample from a Gaussian process defined over the joint context-action space, and develop CGP-UCB, an intuitive upper-confidence style algorithm. We show that by mixing and matching kernels for contexts and actions, CGP-UCB can handle a variety of practical applications. We further provide generic tools for deriving regret bounds when using such composite kernel functions. Lastly, we evaluate our algorithm on two case studies, in the context of automated vaccine design and sensor management. We show that context-sensitive optimization outperforms no or naive use of context. 1
Keyphrases
contextual gaussian process bandit optimization gaussian process generic tool payoff function experimental condition composite kernel function automated vaccine design complex system account uncontrollable environmental condition case study mean payoff function practical application regret bound context-sensitive optimization sensor management key challenge contextual bandit problem naive use intuitive upper-confidence style algorithm joint context-action space relevant document context-action space
