Results 1  10
of
14
Bayesian variable selection in clustering highdimensional data
 Journal of the American Statistical Association
, 2005
"... Over the last decade, technological advances have generated an explosion of data with substantially smaller sample size relative to the number of covariates ( p ≫ n). A common goal in the analysis of such data involves uncovering the group structure of the observations and identifying the discrimina ..."
Abstract

Cited by 34 (4 self)
 Add to MetaCart
Over the last decade, technological advances have generated an explosion of data with substantially smaller sample size relative to the number of covariates ( p ≫ n). A common goal in the analysis of such data involves uncovering the group structure of the observations and identifying the discriminating variables. In this article we propose a methodology for addressing these problems simultaneously. Given a set of variables, we formulate the clustering problem in terms of a multivariate normal mixture model with an unknown number of components and use the reversiblejump Markov chain Monte Carlo technique to define a sampler that moves between different dimensional spaces. We handle the problem of selecting a few predictors among the prohibitively vast number of variable subsets by introducing a binary exclusion/inclusion latent vector, which gets updated via stochastic search techniques. We specify conjugate priors and exploit the conjugacy by integrating out some of the parameters. We describe strategies for posterior inference and explore the performance of the methodology with simulated and real datasets.
Estimating the integrated likelihood via posterior simulation using the harmonic mean identity
 Bayesian Statistics
, 2007
"... The integrated likelihood (also called the marginal likelihood or the normalizing constant) is a central quantity in Bayesian model selection and model averaging. It is defined as the integral over the parameter space of the likelihood times the prior density. The Bayes factor for model comparison a ..."
Abstract

Cited by 24 (2 self)
 Add to MetaCart
The integrated likelihood (also called the marginal likelihood or the normalizing constant) is a central quantity in Bayesian model selection and model averaging. It is defined as the integral over the parameter space of the likelihood times the prior density. The Bayes factor for model comparison and Bayesian testing is a ratio of integrated likelihoods, and the model weights in Bayesian model averaging are proportional to the integrated likelihoods. We consider the estimation of the integrated likelihood from posterior simulation output, aiming at a generic method that uses only the likelihoods from the posterior simulation iterations. The key is the harmonic mean identity, which says that the reciprocal of the integrated likelihood is equal to the posterior harmonic mean of the likelihood. The simplest estimator based on the identity is thus the harmonic mean of the likelihoods. While this is an unbiased and simulationconsistent estimator, its reciprocal can have infinite variance and so it is unstable in general. We describe two methods for stabilizing the harmonic mean estimator. In the first one, the parameter space is reduced in such a way that the modified estimator involves a harmonic mean of heaviertailed densities, thus resulting in a finite variance estimator. The resulting
Shotgun stochastic search for “large p” regression
 Journal of the American Statistical Association
, 2007
"... Model search in regression with very large numbers of candidate predictors raises challenges for both model specification and computation, and standard approaches such as Markov chain Monte Carlo (MCMC) and stepwise methods are often infeasible or ineffective. We describe a novel shotgun stochastic ..."
Abstract

Cited by 17 (3 self)
 Add to MetaCart
Model search in regression with very large numbers of candidate predictors raises challenges for both model specification and computation, and standard approaches such as Markov chain Monte Carlo (MCMC) and stepwise methods are often infeasible or ineffective. We describe a novel shotgun stochastic search (SSS) approach that explores “interesting” regions of the resulting, very highdimensional model spaces to quickly identify regions of high posterior probability over models. We describe algorithmic and modeling aspects, priors over the model space that induce sparsity and parsimony over and above the traditional dimension penalization implicit in Bayesian and likelihood analyses, and parallel computation using cluster computers. We discuss an example from gene expression cancer genomics, comparisons with MCMC and other methods, and theoretical and simulationbased aspects of performance characteristics in largescale regression model search. We also provide software implementing the methods.
Bayesian Partitioning for Classification and Regression
, 1999
"... In this paper we propose a new Bayesian approach to data modelling. The Bayesian partition model constructs arbitrarily complex regression and classification surfaces by splitting the design space into an unknown number of disjoint regions. Within each region the data is assumed to be exchangeable a ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
In this paper we propose a new Bayesian approach to data modelling. The Bayesian partition model constructs arbitrarily complex regression and classification surfaces by splitting the design space into an unknown number of disjoint regions. Within each region the data is assumed to be exchangeable and to come from some simple distribution. Using conjugate priors the marginal likelihoods of the models can be obtained analytically for any proposed partitioning of the space where the number and location of the regions is assumed unknown a priori. Markov chain Monte Carlo simulation techniques are used to obtain distributions on partition structures and by averaging across samples smooth prediction surfaces are formed.
A two component model for counts of infectious diseases
 Biostatistics
, 2006
"... We propose a stochastic model for the analysis of time series of disease counts as collected in typical surveillance systems on notifiable infectious diseases. The model is based on a Poisson or negative binomial observation model with two components: A parameterdriven component relates the disease ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
We propose a stochastic model for the analysis of time series of disease counts as collected in typical surveillance systems on notifiable infectious diseases. The model is based on a Poisson or negative binomial observation model with two components: A parameterdriven component relates the disease incidence to latent parameters describing endemic seasonal patterns, which are typical for infectious disease surveillance data. A observationdriven or epidemic component is modeled with an autoregression on the number of cases at the previous time points. The autoregressive parameter is allowed to change over time according to a Bayesian changepoint model with unknown number of changepoints. Parameter estimates are obtained through Bayesian model averaging using Markov chain Monte Carlo (MCMC) techniques. In analyses of simulated and real datasets we obtain promising results.
Easy Estimation of Normalizing Constants and Bayes Factors from Posterior Simulation: Stabilizing the Harmonic Mean Estimator
, 2000
"... The Bayes factor is a useful summary for model selection. Calculation of this measure involves evaluating the integrated likelihood (or prior predictive density), which can be estimated from the output of MCMC and other posterior simulation methods using the harmonic mean estimator. While this is a ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
The Bayes factor is a useful summary for model selection. Calculation of this measure involves evaluating the integrated likelihood (or prior predictive density), which can be estimated from the output of MCMC and other posterior simulation methods using the harmonic mean estimator. While this is a simulationconsistent estimator, it can have innite variance. In this article we describe a method to stabilize the harmonic mean estimator. Under this approach, the parameter space is reduced such that the modied estimator involves a harmonic mean of heavier tailed densities, thus resulting in a nite variance estimator. We discuss general conditions under which this reduction is applicable and illustrate the proposed method through several examples. Keywords: Bayes factor, Betabinomial, Integrated likelihood, PoissonGamma distribution, Statistical genetics, Variance reduction. Contents 1 Introduction 1 2 Stabilizing the Harmonic Mean Estimator 2 3 Statistical Genetics 6 4 Beta{Binom...
Selection of variables for cluster analysis and classication rules
 J. Amer. Stat. Assoc
, 2008
"... In this article we introduce two procedures for variable selection in cluster analysis and classification rules. One is mainly aimed at detecting the “noisy ” noninformative variables, while the other also deals with multicolinearity and general dependence. Both methods are designed to be used after ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
In this article we introduce two procedures for variable selection in cluster analysis and classification rules. One is mainly aimed at detecting the “noisy ” noninformative variables, while the other also deals with multicolinearity and general dependence. Both methods are designed to be used after a “satisfactory ” grouping procedure has been carried out. A forward–backward algorithm is proposed to make such procedures feasible in large datasets. A small simulation is performed and some real data examples are analyzed.
Extending the Akaike information criterion to mixture regression models
 J. Am. Statist. Assoc
, 2007
"... We examine the problem of jointly selecting the number of components and variables in finite mixture regression models. We find that the Akaike information criterion is unsatisfactory for this purpose because it overestimates the number of components, which in turn results in incorrect variables bei ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
We examine the problem of jointly selecting the number of components and variables in finite mixture regression models. We find that the Akaike information criterion is unsatisfactory for this purpose because it overestimates the number of components, which in turn results in incorrect variables being retained in the model. Therefore, we derive a new information criterion, the mixture regression criterion (MRC), that yields marked improvement in model selection due to what we call the “clustering penalty function. ” Moreover, we prove the asymptotic efficiency of the MRC. We show that it performs well in Monte Carlo studies for the same or different covariates across components with equal or unequal sample sizes. We also present an empirical example on sales territory management to illustrate the application and efficacy of the MRC. Finally, we generalize the MRC to mixture quasilikelihood and mixture autoregressive models, thus extending its applicability to nonGaussian models, discrete responses, and dependent data.
Selection of variables for cluster analysis and classification rules
"... “Selection of variables for cluster analysis and classification rules” ..."