## A Unifying Information-theoretic Framework for Independent Component Analysis (1999)

Citations: | 82 - 8 self |

### BibTeX

@MISC{Lee99aunifying,

author = {Te-won Lee and Mark Girolami and Anthony J. Bell and Terrence J. Sejnowski},

title = {A Unifying Information-theoretic Framework for Independent Component Analysis},

year = {1999}

}

### Years of Citing Articles

### OpenURL

### Abstract

We show that different theories recently proposed for Independent Component Analysis (ICA) lead to the same iterative learning algorithm for blind separation of mixed independent sources. We review those theories and suggest that information theory can be used to unify several lines of research. Pearlmutter and Parra (1996) and Cardoso (1997) showed that the infomax approach of Bell and Sejnowski (1995) and the maximum likelihood estimation approach are equivalent. We show that negentropy maximization also has equivalent properties and therefore all three approaches yield the same learning rule for a fixed nonlinearity. Girolami and Fyfe (1997a) have shown that the nonlinear Principal Component Analysis (PCA) algorithm of Karhunen and Joutsensalo (1994) and Oja (1997) can also be viewed from information-theoretic principles since it minimizes the sum of squares of the fourth-order marginal cumulants and therefore approximately minimizes the mutual information (Comon, 1994). Lambert (19...