DMCA
Scaling Nonparametric Bayesian Inference via Subsample-Annealing
BibTeX
@MISC{Obermeyer_scalingnonparametric,
author = {Fritz Obermeyer and Jonathan Glidden and Eric Jonas},
title = {Scaling Nonparametric Bayesian Inference via Subsample-Annealing},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
We describe an adaptation of the simulated annealing algorithm to nonparametric clus-tering and related probabilistic models. This new algorithm learns nonparamet-ric latent structure over a growing and constantly churning subsample of training data, where the portion of data subsampled can be interpreted as the inverse tempera-ture β(t) in an annealing schedule. Gibbs sampling at high temperature (i.e., with a very small subsample) can more quickly explore sketches of the final latent state by (a) making longer jumps around latent space (as in block Gibbs) and (b) lower-ing energy barriers (as in simulated an-nealing). We prove subsample annealing speeds up mixing time N2 → N in a simple clustering model and exp(N) → N in an-other class of models, where N is data size. Empirically subsample-annealing outper-forms naive Gibbs sampling in accuracy-per-wallclock time, and can scale to larger datasets and deeper hierarchical models. We demonstrate improved inference on million-row subsamples of US Census data and network log data and a 307-row hos-pital rating dataset, using a Pitman-Yor generalization of the Cross Categorization model. 1
Keyphrases
nonparametric bayesian inference u census data high temperature nonparametric clus-tering nonparamet-ric latent structure 307-row hos-pital rating dataset subsample annealing simple clustering model lower-ing energy barrier million-row subsamples block gibbs annealing schedule mixing time n2 network log data an-other class simulated an-nealing training data related probabilistic model data size simulated annealing algorithm pitman-yor generalization small subsample subsample-annealing outper-forms naive gibbs sampling accuracy-per-wallclock time latent space final latent state cross categorization model hierarchical model new algorithm inverse tempera-ture
