Results 11  20
of
97
AN IMPROVED MERGESPLIT SAMPLER FOR CONJUGATE DIRICHLET PROCESS MIXTURE MODELS
, 2003
"... The Gibbs sampler is the standard Markov chain Monte Carlo sampler for drawing samples from the posterior distribution of conjugate Dirichlet process mixture models. Researchers have noticed the Gibbs sampler’s tendency to get stuck in local modes and, thus, poorly explore the posterior distribution ..."
Abstract

Cited by 15 (2 self)
 Add to MetaCart
The Gibbs sampler is the standard Markov chain Monte Carlo sampler for drawing samples from the posterior distribution of conjugate Dirichlet process mixture models. Researchers have noticed the Gibbs sampler’s tendency to get stuck in local modes and, thus, poorly explore the posterior distribution. Jain and Neal (2004) proposed a mergesplit sampler in which a naive random split is sweetened by a series of restricted Gibbs scans, where the number of Gibbs scans is a tuning parameter that must be supplied by the user. In this work, I propose an alternative mergesplit sampler borrowing ideas from sequential importance sampling. My sampler proposes splits by sequentially allocating observations to one of two split components using allocation probabilities that are conditional on previously allocated data. The algorithm does not require further sweetening and is, hence, computationally efficient. In addition, no tuning parameter needs to be chosen. While the conditional allocation of observations is similar to sequential importance sampling, the output from the sampler has the correct stationary distribution due to the use of the MetropolisHastings ratio. The computational efficiency of my sequentiallyallocated mergesplit (SAMS) sampler is
Computing Normalizing Constants for Finite Mixture Models via Incremental Mixture Importance Sampling (IMIS)
, 2003
"... We propose a method for approximating integrated likelihoods in finite mixture models. We formulate the model in terms of the unobserved group memberships, z, and make them the variables of integration. The integral is then evaluated using importance sampling over the z. We propose an adaptive imp ..."
Abstract

Cited by 14 (5 self)
 Add to MetaCart
We propose a method for approximating integrated likelihoods in finite mixture models. We formulate the model in terms of the unobserved group memberships, z, and make them the variables of integration. The integral is then evaluated using importance sampling over the z. We propose an adaptive importance sampling function which is itself a mixture, with two types of component distributions, one concentrated and one diffuse. The more concentrated type of component serves the usual purpose of an importance sampling function, sampling mostly group assignments of high posterior probability. The less concentrated type of component allows for the importance sampling function to explore the space in a controlled way to find other, unvisited assignments with high posterior probability. Components are added adaptively, one at a time, to cover areas of high posterior probability not well covered by the current important sampling function. The method is called Incremental Mixture Importance Sampling (IMIS). IMIS is easy to implement and to monitor for convergence. It scales easily for higher dimensional
Iterated Importance Sampling in Missing Data Problems
, 2005
"... Missing variable models are typical benchmarks for new computational techniques in that the illposed nature of missing variable models o#er a challenging testing ground for these techniques. This was the case for the EM algorithm and the Gibbs sampler, and this is also true for importance sampling ..."
Abstract

Cited by 13 (4 self)
 Add to MetaCart
Missing variable models are typical benchmarks for new computational techniques in that the illposed nature of missing variable models o#er a challenging testing ground for these techniques. This was the case for the EM algorithm and the Gibbs sampler, and this is also true for importance sampling schemes. A population Monte Carlo scheme taking advantage of the latent structure of the problem is proposed. The potential of this approach and its specifics in missing data problems are illustrated in settings of increasing di#culty, in comparison with existing approaches. The improvement brought by a general RaoBlackwellisation technique is also discussed.
Bayesian correlation estimation
, 2004
"... We propose prior probability models for variancecovariance matrices in order to address two important issues. First, the models allow a researcher to represent substantive prior information about the strength of correlations among a set of variables. Secondly, even in the absence of such informatio ..."
Abstract

Cited by 12 (0 self)
 Add to MetaCart
We propose prior probability models for variancecovariance matrices in order to address two important issues. First, the models allow a researcher to represent substantive prior information about the strength of correlations among a set of variables. Secondly, even in the absence of such information, the increased flexibility of the models mitigates dependence on strict parametric assumptions in standard prior models. For example, the model allows a posteriori different levels of uncertainty about correlations among different subsets of variables. We achieve this by including a clustering mechanism in the prior probability model. Clustering is with respect to variables and pairs of variables. Our approach leads to shrinkage towards a mixture structure implied by the clustering. We discuss appropriate posterior simulation schemes to implement posterior inference in the proposed models, including the evaluation of normalising constants that are functions of parameters of interest. The normalising constants result from the restriction that the correlation matrix be positive definite. We discuss examples based on simulated data, a stock return dataset and a population genetics dataset.
Modelbased clustering of multiple time series
 CEPR Discussion Paper
, 2004
"... We propose to use the attractiveness of pooling relatively short time series that display similar dynamics, but without restricting to pooling all into one group. We suggest to estimate the appropriate grouping of time series simultaneously along with the groupspecific model parameters. We cast est ..."
Abstract

Cited by 12 (0 self)
 Add to MetaCart
We propose to use the attractiveness of pooling relatively short time series that display similar dynamics, but without restricting to pooling all into one group. We suggest to estimate the appropriate grouping of time series simultaneously along with the groupspecific model parameters. We cast estimation into the Bayesian framework and use Markov chain Monte Carlo simulation methods. We discuss model identification and base model selection on marginal likelihoods. A simulation study documents the efficiency gains in estimation and forecasting that are realized when appropriately grouping the time series of a panel. Two economic applications illustrate the usefulness of the method in analyzing also extensions to Markov switching within clusters and heterogeneity within clusters, respectively. JEL classification: C11,C33,E32
SequentiallyAllocated MergeSplit Sampler for Conjugate and Nonconjugate Dirichlet Process Mixture Models
, 2005
"... This paper proposes a new efficient mergesplit sampler for both conjugate and nonconjugate Dirichlet process mixture (DPM) models. These Bayesian nonparametric models are usually fit using Markov chain Monte Carlo (MCMC) or sequential importance sampling (SIS). The latest generation of Gibbs and Gi ..."
Abstract

Cited by 12 (0 self)
 Add to MetaCart
This paper proposes a new efficient mergesplit sampler for both conjugate and nonconjugate Dirichlet process mixture (DPM) models. These Bayesian nonparametric models are usually fit using Markov chain Monte Carlo (MCMC) or sequential importance sampling (SIS). The latest generation of Gibbs and Gibbslike samplers for both conjugate and nonconjugate DPM models effectively update the model parameters, but can have difficulty in updating the clustering of the data. To overcome this deficiency, mergesplit samplers have been developed, but until now these have been limited to conjugate or conditionallyconjugate DPM models. This paper proposes a new MCMC sampler, called the sequentiallyallocated mergesplit (SAMS) sampler. The sampler borrows ideas from sequential importance sampling. Splits are proposed by sequentially allocating observations to one of two split components using allocation probabilities that condition on previously allocated data. The SAMS sampler is applicable to general nonconjugate DPM models as well as conjugate models. Further, the proposed sampler is substantially more efficient than existing conjugate and nonconjugate samplers.
A Constrained SemiSupervised Learning Approach to Data Association
 In European Conference for Computer Vision (ECCV
, 2004
"... Data association (obtaining correspondences) is a ubiquitous problem in computer vision. It appears when matching image features across multiple images, matching image features to object recognition models and matching image features to semantic concepts. In this paper, we show how a wide class of d ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
Data association (obtaining correspondences) is a ubiquitous problem in computer vision. It appears when matching image features across multiple images, matching image features to object recognition models and matching image features to semantic concepts. In this paper, we show how a wide class of data association tasks arising in computer vision can be interpreted as a constrained semisupervised learning problem. This interpretation opens up room for the development of new, more efficient data association methods. In particular, it leads to the formulation of a new principled probabilistic model for constrained semisupervised learning that accounts for uncertainty in the parameters and missing data. By adopting an ingenious data augmentation strategy, it becomes possible to develop an efficient MCMC algorithm where the highdimensional variables in the model can be sampled efficiently and directly from their posterior distributions. We demonstrate the new model and algorithm on synthetic data and the complex problem of matching image features to words in the image captions.
Contemplating evidence: properties, extensions of, and alternatives to nested sampling
, 2007
"... Nested sampling is a novel simulation method for approximating marginal likelihoods, proposed by Skilling (2007a,b). We establish that nested sampling leads to an error that vanishes at the standard Monte Carlo rate N −1/2, where N is a tuning parameter that is proportional to the computational effo ..."
Abstract

Cited by 11 (10 self)
 Add to MetaCart
Nested sampling is a novel simulation method for approximating marginal likelihoods, proposed by Skilling (2007a,b). We establish that nested sampling leads to an error that vanishes at the standard Monte Carlo rate N −1/2, where N is a tuning parameter that is proportional to the computational effort, and that this error is asymptotically Gaussian. We show that the corresponding asymptotic variance typically grows linearly with the dimension of the parameter. We use these results to discuss the applicability and efficiency of nested sampling in realistic problems, including posterior distributions for mixtures. We propose an extension of nested sampling that makes it possible to avoid resorting to MCMC to obtain the simulated points. We study two alternative methods for computing marginal likelihood, which, in contrast with nested sampling, are based on draws from the posterior distribution and we conduct a comparison with nested sampling on several realistic examples.
Bayesian finite mixtures with an unknown number of components: the allocation sampler
 University of Glasgow
, 2005
"... A new Markov chain Monte Carlo method for the Bayesian analysis of finite mixture distributions with an unknown number of components is presented. The sampler is characterized by a state space consisting only of the number of components and the latent allocation variables. Its main advantage is that ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
A new Markov chain Monte Carlo method for the Bayesian analysis of finite mixture distributions with an unknown number of components is presented. The sampler is characterized by a state space consisting only of the number of components and the latent allocation variables. Its main advantage is that it can be used, with minimal changes, for mixtures of components from any parametric family, under the assumption that the component parameters can be integrated out of the model analytically. Artificial and real data sets are used to illustrate the method and mixtures of univariate and of multivariate normals are explicitly considered. The problem of label switching, when parameter inference is of interest, is addressed in a postprocessing stage.
Learning to Recognize Objects with Little Supervision
, 2008
"... This paper shows (i) improvements over stateoftheart local feature recognition systems, (ii) how to formulate principled models for automatic local feature selection in object class recognition when there is little supervised data, and (iii) how to formulate sensible spatial image context models ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
This paper shows (i) improvements over stateoftheart local feature recognition systems, (ii) how to formulate principled models for automatic local feature selection in object class recognition when there is little supervised data, and (iii) how to formulate sensible spatial image context models using a conditional random field for integrating local features and segmentation cues (superpixels). By adopting sparse kernel methods, Bayesian learning techniques and data association with constraints, the proposed model identifies the most relevant sets of local features for recognizing object classes, achieves performance comparable to the fully supervised setting, and obtains excellent results for image classification.