Abstract

We introduce the independent factor analysis (IFA) method for recovering independent hidden sources from their observed mixtures. IFA generalizes and unifies ordinary factor analysis (FA), principal component analysis (PCA), and independent component analysis (ICA), and can handle not only square noiseless mixing, but also the general case where the number of mixtures differs from the number of sources and the data are noisy. IFA is a two-step procedure. In the first step, the source densities, mixing matrix and noise covariance are estimated from the observed data by maximum likelihood. For this purpose we present an expectation-maximization (EM) algorithm, which performs unsupervised learning of an associated probabilistic model of the mixing situation. Each source in our model is described by a mixture of Gaussians, thus all the probabilistic calculations can be performed analytically. In the second step, the sources are reconstructed from the observed data by an optimal non-linear ...

Citations

6514	Elements of Information Theory - Cover, Thomas - 1991
6231	Maximum likelihood from incomplete data via the EM algorithm - Dempster, Laird - 1977
1034	Independent component analysis: A new concept - COMON - 1994
969	Generalized Additive Models - Hastie, Tibshirani - 1990
830	An information-maximization approach to blind separation and blind deconvolution - Bell, Sejnowski - 1995
683	Emergence of simplecell receptive field properties by learning a sparse code for natural images. Nature 381 - OLSHAUSEN, FIELD - 1996
634	Hierarchical mixtures of experts and the EM algorithm - Jordan, Jacobs - 1994
612	A view of the EM algorithm that justifies incremental, sparse, and other variants - Neal, Hinton - 1998
429	A new learning algorithm for blind signal separation - SI, Cichoki, et al. - 1996
427	Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research 37 - OLSHAUSEN, FIELD - 1997
417	Bayesian interpolation - MacKay - 1992
368	Blind separation of sources, part i: An adaptive algorithm based on neuromimetic architecture - Jutten, Herault - 1991
359	Probabilistic principal component analysis - Tipping, Bishop - 1999
327	Projection pursuit regression - Friedman, Stuetzle - 1981
325	Equivariant adaptative source separation - Cardoso, Labeld - 1996
312	A fast fixed-point algorithm for independent component analysis - Hyvärinen, Oja - 1997
234	GTM: The Generative Topographic Mapping - Bishop, Svensen, et al. - 1998
208	A unifying review of linear Gaussian models - Roweis - 1999
190	The wake-sleep algorithm for unsupervised neural networks - Hinton, Dayan, et al. - 1995
189	Projection pursuit - HUBER - 1985
165	The Helmholtz machine - Dayan, Hinton, et al. - 1995
113	An Introduction to Latent Variable Models - Everitt - 1984
102	Mean field theory for sigmoid belief networks - Zemel, Saul, et al. - 1996
101	Maximum likelihood and covariant algorithms for independent component analysis - MacKay - 1996
98	Autoencoders, minimum description length, and Helmholtz free energy - Hinton, Zemel - 1994
82	Em algorithms for pca and spca - Roweis - 1998
74	EM algorithms for ML factor analysis - Rubin, Thayer - 1982
60	Maximum likelihood blind source separation: A context-sensitive generalization - Pearlmutter, Parra - 1997
58	Infomax and maximum likelihood for source separation - Cardoso - 1997
58	Adaptive elastic models for hand-printed character recognition - Hinton, Williams, et al. - 1992
54	Blind separation of delayed and convolved sources - Lee, Bell, et al. - 1996
53	Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models - Moulines, Cardoso, et al. - 1997
49	Factorial learning and the EM algorithm - Ghahramani - 1994
47	Blind separation of sources - Comon, Jutten, et al. - 1991
45	separation of instantaneous mixture of sources via an independent component analysis - Pham, “Blind - 1996
36	Learning non-linear overcomplete representations for efficient coding - LEWICKI, SEJNOWSKI - 1998
32	source separation and deconvolution: The dynamic component analysis algorithms - Attias, Schreiner, et al. - 1998
30	Inferring sparse, overcomplete image codes using an efficient coding framework - Lewicki, Olshausen - 1998
24	Maximum likelihood source separation for discrete sources - Belouchrani, Cardoso - 1994
21	Learning Linear, Sparse, Factorial Codes - Olshausen - 1996
9	Factorial hidden Markov models, Machine learning 29 - Ghahramani, Jordan - 1997
5	Exploiting tractable structures in intractable networks - Saul, Jordan - 1995