Results 1  10
of
28
An informationmaximization approach to blind separation and blind deconvolution
 NEURAL COMPUTATION
, 1995
"... ..."
Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources
, 1999
"... An extension of the infomax algorithm of Bell and Sejnowski (1995) is presented that is able blindly to separate mixed signals with sub and supergaussian source distributions. This was achieved by using a simple type of learning rule first derived by Girolami (1997) by choosing negentropy as a proj ..."
Abstract

Cited by 233 (21 self)
 Add to MetaCart
An extension of the infomax algorithm of Bell and Sejnowski (1995) is presented that is able blindly to separate mixed signals with sub and supergaussian source distributions. This was achieved by using a simple type of learning rule first derived by Girolami (1997) by choosing negentropy as a projection pursuit index. Parameterized probability distributions that have sub and supergaussian regimes were used to derive a general learning rule that preserves the simple architecture proposed by Bell and Sejnowski (1995), is optimized using the natural gradient by Amari (1998), and uses the stability analysis of Cardoso and Laheld (1996) to switch between sub and supergaussian regimes. We demonstrate that the extended infomax algorithm is able to separate 20 sources with a variety of source distributions easily. Applied to highdimensional data from electroencephalographic recordings, it is effective at separating artifacts such as eye blinks and line noise from weaker electrical signals that arise from sources in the brain.
Removing Electroencephalographic Artifacts: Comparison between ICA and PCA
, 1998
"... Pervasive electroencephalographic (EEG) artifacts associated with blinks, eyemovements, muscle noise, cardiac signals, and line noise poses a major challenge for EEG interpretation and analysis. Here, we propose a generally applicable method for removing a wide variety of artifacts from EEG records ..."
Abstract

Cited by 144 (21 self)
 Add to MetaCart
Pervasive electroencephalographic (EEG) artifacts associated with blinks, eyemovements, muscle noise, cardiac signals, and line noise poses a major challenge for EEG interpretation and analysis. Here, we propose a generally applicable method for removing a wide variety of artifacts from EEG records based on an extended version of an Independent Component Analysis (ICA) algorithm [2, 12] for performing blind source separation on linear mixtures of independent source signals. Our results show that ICA can effectively separate and remove contamination from a wide variety of artifactual sources in EEG records with results comparing favorably to those obtained using Principal Component Analysis. 1 INTRODUCTION Since the landmark development of electroencephalography (EEG) in 1928 by Berger, scalp EEG has been used as a clinical tool for the diagnosis and treatment of brain diseases, and used as a noninvasive approach for research in the quantitative study of human neurophysiology. Ironic...
A Unifying Informationtheoretic Framework for Independent Component Analysis
, 1999
"... We show that different theories recently proposed for Independent Component Analysis (ICA) lead to the same iterative learning algorithm for blind separation of mixed independent sources. We review those theories and suggest that information theory can be used to unify several lines of research. Pea ..."
Abstract

Cited by 91 (8 self)
 Add to MetaCart
We show that different theories recently proposed for Independent Component Analysis (ICA) lead to the same iterative learning algorithm for blind separation of mixed independent sources. We review those theories and suggest that information theory can be used to unify several lines of research. Pearlmutter and Parra (1996) and Cardoso (1997) showed that the infomax approach of Bell and Sejnowski (1995) and the maximum likelihood estimation approach are equivalent. We show that negentropy maximization also has equivalent properties and therefore all three approaches yield the same learning rule for a fixed nonlinearity. Girolami and Fyfe (1997a) have shown that the nonlinear Principal Component Analysis (PCA) algorithm of Karhunen and Joutsensalo (1994) and Oja (1997) can also be viewed from informationtheoretic principles since it minimizes the sum of squares of the fourthorder marginal cumulants and therefore approximately minimizes the mutual information (Comon, 1994). Lambert (19...
Blind Separation of Delayed and Convolved Sources
, 1997
"... We address the difficult problem of separating multiple speakers with multiple microphones in a real room. We combine the work of Torkkola and Amari, Cichocki and Yang, to give Natural Gradient information maximisation rules for recurrent (IIR) networks, blindly adjusting delays, separating and deco ..."
Abstract

Cited by 67 (1 self)
 Add to MetaCart
We address the difficult problem of separating multiple speakers with multiple microphones in a real room. We combine the work of Torkkola and Amari, Cichocki and Yang, to give Natural Gradient information maximisation rules for recurrent (IIR) networks, blindly adjusting delays, separating and deconvolving mixed signals. While they work well on simulated data, these rules fail in real rooms which usually involve nonminimum phase transfer functions, notinvertible using stable IIR filters. An approach that sidesteps this problem is to perform infomax on a feedforward architecture in the frequency domain (Lambert 1996). We demonstrate realroom separation of two natural signals using this approach. 1 The problem. In the linear blind signal processing problem ([3, 2] and references therein), N signals, s(t) = [s 1 (t) : : : s N (t)] T , are transmitted through a medium so that an array of N sensors picks up a set of signals x(t) = [x 1 (t) : : : xN (t)] T , each of which has bee...
Source Separation of Real World Signals
 In Proc. I.E.E.E / I.C.N.N, International Conference on Neural Networks, Vol 4, pp 2129 – 2134
, 1997
"... ..."
Survey of Sparse and NonSparse Methods in Source Separation
, 2005
"... Source separation arises in a variety of signal processing applications, ranging from speech processing to medical image analysis. The separation of a superposition of multiple signals is accomplished by taking into account the structure of the mixing process and by making assumptions about the sour ..."
Abstract

Cited by 41 (1 self)
 Add to MetaCart
Source separation arises in a variety of signal processing applications, ranging from speech processing to medical image analysis. The separation of a superposition of multiple signals is accomplished by taking into account the structure of the mixing process and by making assumptions about the sources. When the information about the mixing process and sources is limited, the problem is called ‘blind’. By assuming that the sources can be represented sparsely in a given basis, recent research has demonstrated that solutions to previously problematic blind source separation problems can be obtained. In some cases, solutions are possible to problems intractable by previous nonsparse methods. Indeed, sparse methods provide a powerful approach to the separation of linear mixtures of independent data. This paper surveys the recent arrival of sparse blind source separation methods and the previously existing nonsparse methods, providing insights and appropriate hooks into the literature along the way.
Factorial coding of natural images: how effective are linear models in removing higherorder dependencies?
 JOURNAL OF THE OPTICAL SOCIETY OF AMERICA A
, 2006
"... The performance of unsupervised learning models for natural images is evaluated quantitatively by means of information theory. We estimate the gain in statistical independence (the multiinformation reduction) achieved with independent component analysis (ICA), principal component analysis (PCA), z ..."
Abstract

Cited by 24 (8 self)
 Add to MetaCart
The performance of unsupervised learning models for natural images is evaluated quantitatively by means of information theory. We estimate the gain in statistical independence (the multiinformation reduction) achieved with independent component analysis (ICA), principal component analysis (PCA), zerophase whitening, and predictive coding. Predictive coding is translated into the transform coding framework, where it can be characterized by the constraint of a triangular filter matrix. A randomly sampled whitening basis and the Haar wavelet are included into the comparison as well. The comparison of all these methods is carried out for different patch sizes, ranging from 2x2 to 16x16 pixels. In spite of large differences in the shape of the basis functions, we find only small differences in the multiinformation between all decorrelation transforms (5% or less) for all patch sizes. Among the secondorder methods, PCA is optimal for small patch sizes and predictive coding performs best for large patch sizes. The extra gain achieved with ICA is always less than 2%. In conclusion, the `edge filters&amp;amp;amp;amp;lsquo; found with ICA lead only to a surprisingly small improvement in terms of its actual objective.
Fast Blind Separation Based on Information Theory.
 in Proc. Intern. Symp. on Nonlinear Theory and Applications (NOLTA), Las Vegas
, 1995
"... Blind separation is an information theoretic problem, and we have proposed an information theoretic `sigmoidbased' solution [2]. Here we elaborate on several aspects of that solution. Firstly, we argue that the separation matrix may be exactly found by maximising the joint entropy of the rando ..."
Abstract

Cited by 19 (5 self)
 Add to MetaCart
Blind separation is an information theoretic problem, and we have proposed an information theoretic `sigmoidbased' solution [2]. Here we elaborate on several aspects of that solution. Firstly, we argue that the separation matrix may be exactly found by maximising the joint entropy of the random vector resulting from a linear transformation of the mixtures followed by sigmoidal nonlinearities which are the cumulative density functions of the `unknown' sources. Secondly, we present the learning rule for performing this maximisation. Thirdly, we discuss the role of prior knowledge of the c.d.f.'s of the sources in customising the learning rule. We argue that sigmoidbased methods are better able to make use of this prior knowledge than cumulantbased methods, because the optimal nonlinearity they should use is just an estimate of the source c.d.f. We also suggest that they may have the edge in terms of robustness and speed of convergence. Improvements in convergence speed have been fac...
Efficient Source Adaptivity in Independent Component Analysis
, 2001
"... A basic element in most ICA algorithms is the choice of a model for the score functions of the unknown sources. While this is usually based on approximations, for large data sets it is possible to achieve `source adaptivity' by directly estimating from the data the `true' score functions o ..."
Abstract

Cited by 15 (1 self)
 Add to MetaCart
A basic element in most ICA algorithms is the choice of a model for the score functions of the unknown sources. While this is usually based on approximations, for large data sets it is possible to achieve `source adaptivity' by directly estimating from the data the `true' score functions of the sources. In this paper we describe an efficient scheme for achieving this by extending the fast density estimation method of Silverman (1982). We show with a real and a synthetic experiment that our method can provide more accurate solutions than stateoftheart methods when optimization is carried out in the vicinity of the global minimum of the contrast function.