Abstract

A central issue in principal component analysis (PCA) is choosing the number of principal components to be retained. By interpreting PCA as density estimation, we show how to use Bayesian model selection to estimate the true dimensionality of the data. The resulting estimate is simple to compute yet guaranteed to pick the correct dimensionality, given enough data. The estimate involves an integral over the Steifel manifold of k-frames, which is difficult to compute exactly. But after choosing an appropriate parameterization and applying Laplace's method, an accurate and practical estimator is obtained. In simulations, it is convincingly better than cross-validation and other proposed algorithms, plus it runs much faster.

Citations

476	Probabilistic visual learning for object representation. Early Visual Learning - Moghaddam, Pentland - 1996
359	Probabilistic principal component analysis - Tipping, Bishop - 1999
334	Mixtures of probabilistic principal component analyzers - Tipping, Bishop - 1999
197	G: The EM algorithm for mixtures of factor analyzers - Ghahramani, Hinton - 1997
192	Probabilistic visual learning for object detection - Moghaddam, Pentland - 1995
124	Probable networks and plausible predictions – a review of practical Bayesian methods for supervised neural networks - MacKay - 1995
120	M.: Variational inference for Bayesian mixtures of factor analysers - Ghahramani, Beal - 2000
70	Raftery A: Bayes factors and model uncertainty - Kass - 1995
68	Surface Learning with Applications to Lipreading - Bregler, Omohundro - 1994
52	DR: An algorithm for finding intrinsic dimensionality of data - Fukunaga, Olsen - 1971
37	Bayesian PCA - Bishop - 1999
30	Normal multivariate analysis and the orthogonal group - James - 1954
24	An antipodally symmetric distribution on the sphere - Bingham - 1974
14	Bayesian Modeling of Facial Similarity - Moghaddam, Jebara, et al. - 1999
13	Inferring the eigenvalues of covariance matrices from limited, noisy data - Everson, Roberts - 2000
9	Model order selection for the singular value decomposition and the discrete Karhunen–Loeve transform using a Bayesian approach - Rajan, Rayner - 1997
9	The von Mises–Fisher matrix distribution in orientation statistics - Khatri, Mardia - 1977
3	EM algorithms for - ROWEIS - 1997