Abstract not found

The last ten years have witnessed the development of sampling frameworks that permit the construction of Markov chains which simultaneously traverse both parameter and model space. In this time substantial methodological progress has been made. In this article we present a survey of the current state of the art and evaluate some of the most recent advances in this field. We also discuss future research perspectives in the context of the drive to develop sampling mechanisms with high degrees of both efficiency and automation. 1

Abstract not found

We present full Bayesian analysis of finite mixtures of multivariate normals with unknown number of components. We adopt reversible jump Markov chain Monte Carlo and we construct, in a manner similar to that of Richardson and Green (1997), split and merge moves that produce good mixing of the Markov chains. The split moves are constructed on the space of eigenvectors and eigenvalues of the current covariance matrix so that the proposed covariance matrices are positive definite. Our proposed methodology has applications in classification and discrimination as well as heterogeneity modelling. We test our algorithm with real and simulated data.

In this contribution, we propose a generic online (also sometimes called adaptive or recursive) version of the Expectation-Maximisation (EM) algorithm applicable to latent variable models of independent observations. Compared to the algorithm of Titterington (1984), this approach is more directly connected to the usual EM algorithm and does not rely on integration with respect to the complete data distribution. The resulting algorithm is usually simpler and is shown to achieve convergence to the stationary points of the Kullback-Leibler divergence between the marginal distribution of the observation and the model distribution at the optimal rate, i.e., that of the maximum likelihood estimator. In addition, the proposed approach is also suitable for conditional (or regression) models, as illustrated in the case of the mixture of linear regressions model. Keywords: Latent data models, Expectation-Maximisation, adaptive algorithms, online estimation, stochastic approximation, Polyak-Ruppert averaging, mixture of regressions. 1

in quality

Abstract not found

Let (Fk ) k1 be a nested family of parametric classes of densities with finite Vapnik-Chervonenkis dimension. Let f be a probability density belonging to Fk , where k is the unknown smallest integer such that f 2 Fk . Given a random sample X1 ; : : : ; Xn drawn from f , an integer k0 1 and a real number 2 (0; 1), we introduce a new, simple, explicit -level consistent testing procedure of the null hypothesis fH0 : k = k0g versus the alternative fH1 : k 6= k0g. Our method is inspired by the combinatorial tools developed in Devroye and Lugosi [1] and it includes a wide range of density models, such as mixture models, neural networks or exponential families.

Abstract. In the past ten years there has been a dramatic increase of interest in the Bayesian analysis of finite mixture models. This is primarily because of the emergence of Markov chain Monte Carlo (MCMC) methods. Whilst MCMC provides a convenient way to draw inference from complicated statistical models, there are many, perhaps under appreciated, problems associated with the MCMC analysis of mixtures. The problems are mainly caused by the nonidentifiability of the components under symmetric priors, which leads to so called label switching in the MCMC output. This will mean that ergodic averages of component specific quantities will be identical and thus useless for inference. We review the solutions to the label switching problem, such as artificial identifiability constraints (e.g. Diebolt & Robert (1994)), relabelling algorithms (Stephens 1997a) and label invariant loss functions (Celeux, Hurn & Robert 2000). We also review various MCMC sampling schemes that have been suggested for mixture models and discuss posterior sensitivity to prior specification.

Mixture models have very wide application. However, the problem of determining the number of components, K say, is one of the challenging problems in this area. There are several approaches in the literature for testing K for different values. A recent alternative approach is to treat K as unknown, and to model K and the mix- ture component parameters jointly; see for example Richardson and Green (1997). In this paper, we propose an approach based on a birth-death process. In contrast with the approach in Stephens(2000), we make use of the latent indicators so that the approach can be used to solve the problems with missing data when calculation of the likelihood requires knowledge of the unobservable latent variables. Specifically, we use the method to analyse hidden Markov models with unknown numbers of states. The model and the algorithm are illustrated by some real examples.