## Dealing with label switching in mixture models (2000)

### Cached

### Download Links

Venue: | Journal of the Royal Statistical Society, Series B |

Citations: | 111 - 0 self |

### BibTeX

@ARTICLE{Stephens00dealingwith,

author = {Matthew Stephens},

title = {Dealing with label switching in mixture models},

journal = {Journal of the Royal Statistical Society, Series B},

year = {2000},

volume = {62},

pages = {795--809}

}

### Years of Citing Articles

### OpenURL

### Abstract

In a Bayesian analysis of finite mixture models, parameter estimation and clustering are sometimes less straightforward that might be expected. In particular, the common practice of estimating parameters by their posterior mean, and summarising joint posterior distributions by marginal distributions, often leads to nonsensical answers. This is due to the so-called “labelswitching” problem, which is caused by symmetry in the likelihood of the model parameters. A frequent response to this problem is to remove the symmetry using artificial identifiability constraints. We demonstrate that this fails in general to solve the problem, and describe an alternative class of approaches, relabelling algorithms, which arise from attempting to minimise the posterior expected loss under a class of loss functions. We describe in detail one particularly simple and general relabelling algorithm, and illustrate its success in dealing with the labelswitching problem on two examples.

### Citations

508 | Mixture Densities, Maximum Likelihood and the EM Algorithm - Redner, Walker - 1984 |

443 | On Bayesian analysis of mixtures with an unknown number of components (with discussion
- Richardson, Green
- 1998
(Show Context)
Citation Context ...rior 1 Introduction The so-called label-switching problem arises when taking a Bayesian approach to parameter estimation and clustering using mixture models (see for example Diebolt and Robert, 1994; =-=Richardson and Green, 1997-=-). The term label-switching was used by Redner and Walker (1984) to describe the invariance of the likelihood under relabelling of the mixture components. In a Bayesian context this invariance can lea... |

400 | Bayesian density estimation and inference using mixtures - Escobar, West - 1995 |

326 | Marginal likelihood from the gibbs output - Chib - 1995 |

149 | Bayesian model choice via Markov chain Monte Carlo methods - Carlin, Chib - 1995 |

145 |
Estimation of Finite Mixture Distributions through Bayesian Sampling
- Diebolt, Robert
- 1994
(Show Context)
Citation Context ...re model, Multimodal posterior 1 Introduction The so-called label-switching problem arises when taking a Bayesian approach to parameter estimation and clustering using mixture models (see for example =-=Diebolt and Robert, 1994-=-; Richardson and Green, 1997). The term label-switching was used by Redner and Walker (1984) to describe the invariance of the likelihood under relabelling of the mixture components. In a Bayesian con... |

112 | Computational and inferential difficulties with mixture posterior distributions - Celeux, Hurn, et al. - 2000 |

73 | Bayesian Model Comparison via Jump Diffusions - Phillips, Smith - 1996 |

57 | Density estimation with confidence sets exemplified by superclusters and voids in the galaxies - Roeder - 1990 |

53 |
Bayesian methods for mixtures of normal distributions
- STEPHENS
- 1997
(Show Context)
Citation Context ...�©�§�� ��� sample��� ��§�©¨©¨©�§�� � £ ¥ � � � � �¥£ ¥�� � � � �§¦ ¥ � � � � � � ���¨� � � ���¨� �¨¡ � � � � � � ¡ ¥ , applying permutations such that the constraint is satisfied by the permuted (see =-=Stephens, 1997-=-b, Proposition 3.1, for formal justification). Figure 3 shows estimates of the marginal posterior densities of the means based on the permuted sample. These densities continue to exhibit multimodality... |

51 | Bayesian inference in Hidden Markov models through the reversible jump Markov chain Monte Carlo method
- Robert, Rydén, et al.
- 2000
(Show Context)
Citation Context ...contexts other than finite mixture models in which the label-switching problem occurs, and for which Algorithm 4.2 is inappropriate. For example, in a Bayesian analysis of a Hidden Markov Model (e.g. =-=Robert et al., 2000-=-) the likelihood is invariant under permutation of the labels of the hidden states. If suitably symmetric prior distributions are used then the label-switching problem arises when attempting to infer,... |

19 |
1997a)Discussion on `On Bayesian analysis of mixtures with an unknown number of components' �by
- Stephens
(Show Context)
Citation Context ...hrees¢¡ distributions to the galaxy data, separating the major mode and minor mode by eye as described in the text. 13swhen the number of clusters is allowed to vary (e.g. Richardson and Green, 1997; =-=Stephens, 2000-=-), although in this case problems with genuine multimodality are likely to arise. An on-line algorithm Algorithm 4.2 can be computationally demanding on storage. In our implementation � we � stored ma... |

17 |
Bayesian inference for mixtures: The label-switching problem
- Celeux
- 1998
(Show Context)
Citation Context ...fore, for larger data sets, with more components, or longer runs of the MCMC sampler, an on-line algorithm (along the lines of �¤£¢�£¢¦¥ bytes of storage in our implementation. Both the computational =-=Celeux, 1998-=-) would be preferable. Such an algorithm is given in Section 6. We can use �sthe found using Algorithm 4.2 to cluster the observations into groups by choosing the allocation ��� variables to �s� ¤¨§ �... |

17 | Stochastic version of the em algorithm: An experimental study in the mixture case - Celeux, Chaveau, et al. - 1996 |

17 | 2000)Computational and inferential di culties with mixture posterior distributions - Celeux, Hurn, et al. |

12 | An application of the Laplace method to finite mixture distributions - Crawford - 1994 |

7 | 1990)Density estimation with con®dence sets exempli®ed by superclusters and voids in the galaxies - Roeder |

4 | Contribution to the discussion of paper by Richardson and Green - Celeux - 1997 |

4 |
Contribution to the discussion of paper by Richardson and Green
- Stephens
- 1997
(Show Context)
Citation Context ...�©�§�� ��� sample��� ��§�©¨©¨©�§�� � £ ¥ � � � � �¥£ ¥�� � � � �§¦ ¥ � � � � � � ���¨� � � ���¨� �¨¡ � � � � � � ¡ ¥ , applying permutations such that the constraint is satisfied by the permuted (see =-=Stephens, 1997-=-b, Proposition 3.1, for formal justification). Figure 3 shows estimates of the marginal posterior densities of the means based on the permuted sample. These densities continue to exhibit multimodality... |

3 | 1997)Discussion on `On Bayesian analysis of mixtures with an unknown number of components' �by - Celeux |

2 | 1994)Estimation of ®nite mixture distributions through Bayesian sampling - Diebolt, Robert |

1 | 1998)Bayesian inference for mixtures: the label-switching problem - unknown authors |

1 | 1994)An application of the Laplace method to ®nite mixture distributions - Crawford |

1 | 1997b)Bayesian methods for mixtures of normal distributions. DPhil Thesis - unknown authors |