Blind signal separation (BSS) and independent component analysis (ICA) are emerging techniques of array processing and data analysis, aiming at recovering unobserved signals or `sources' from observed mixtures (typically, the output of an array of sensors), exploiting only the assumption of mutual independence between the signals. The weakness of the assumptions makes it a powerful approach but requires to venture beyond familiar second order statistics. The objective of this paper is to review some of the approaches that have been recently developed to address this exciting problem, to show how they stem from basic principles and how they relate to each other.

An extension of the infomax algorithm of Bell and Sejnowski (1995) is presented that is able to blindly separate mixed signals with sub- and super-Gaussian source distributions. This was achieved by using a simple type of learning rule first derived by Girolami (1997) by choosing negentropy as a projection pursuit index. Parameterized probability distributions that have suband super-Gaussian regimes were used to derive a general learning rule that preserves the simple architecture proposed by Bell and Sejnowski (1995), is optimized using the natural gradient by Amari (1998), and uses the stability analysis of Cardoso and Laheld (1996) to switch between sub- and super-Gaussian regimes. We demonstrate that the extended infomax algorithm is able to easily separate 20 sources with a variety of source distributions. Applied to high-dimensional data from electroencephalographic (EEG) recordings, it is effective at separating artifacts such as eye blinks and line noise from weaker electrical ...

This article considers high-order measures of independence for the independent component analysis problem and discusses the class of Jacobi algorithms for their optimization. Several implementations are discussed. We compare the proposed approaches with gradient-based techniques from the algorithmic point of view and also on a set of biomedical data.

Most ICA algorithms are based on a model of stationary sources. This paper considers exploiting the (possible) non-stationarity of the sources to achieve separation. We introduce two objective functions based on the likelihood and on mutual information in a simple Gaussian non stationary model and we show how they can be optimized, off-line or on-line, by simple yet remarkably efficient algorithms (one is based on a novel joint diagonalization procedure, the other on a Newton-like technique). The paper also includes (limited) numerical experiments and a discussion contrasting non-Gaussian and non-stationary models. 1. INTRODUCTION The aim of this paper is to develop a blind source separation procedure adapted to source signals with time varying intensity (such as speech signals). For simplicity, we shall restrict ourselves to the simplest mixture model: X(t) = AS(t) (1) where X(t) = [X 1 (t) XK (t)] T is the vector of observations (at time t), A is a fixed unknown K K inver...

Introduction An adaptive filter is defined as a self-designing system that relies for its operation on a recursive algorithm, which makes it possible for the filter to perform satisfactorily in an environment where knowledge of the relevant statistics is not available. Adaptive filters are classified into two main groups: linear, and non linear. Linear adaptive filters compute an estimate of a desired response by using a linear combination of the available set of observables applied to the input of the filter. Otherwise, the adaptive filter is said to be nonlinear. Adaptive filters may also be classified into: (i) Supervised adaptive filters, which require the availability of a training sequence that provides different realizations of a desired response for a specified input signal vector. The desired response is compared against the actual response of the filter due to the input signal vector, and the resulting error signal is

In this paper we introduce a novel procedure for separating an instantaneous mixture of source based on the order statistics. The method is derived in a general context of independence component analysis, using a contrast function defined in term of the Kullback-Leibner divergence or of the mutual information. We introduce a discretized form of this contrast permitting its easy estimation through the order statistics. We show that the local contrast property is preserved and also derive a global contrast exploiting only the information of the support of the distribution (in the case this support is finite). Some simulations are given illustrating the good performance of the method. 1 Introduction The problem of separation of sources has been the subject of rapid development in the signal processing literature recently (see for example [2] -- [5], [7] -- [12], [14], [15] : : : ). We consider here the simplest case where one observes K sequences X 1 (t), : : : , XK (t), each being a li...

This paper provides an iterative algorithm to jointly approximately diagonalize K Hermitian positive definite matrices Γ_1, ..., Γ_K . Specifically it calculates the matrix B which minimizes the criterion P K k=1 n k [log det diag(BC k B ) log det(BC k B )], n k being positive numbers, which is a measure of the deviation from diagonality of the matrices BC_k B*. The convergence of the algorithm is discussed and some numerical experiments are performed showing the good performance of the algorithm.

In recent years, several methods have been proposed for the discovery of causal structure from non-experimental data. Such methods make various assumptions on the data generating process to facilitate its identification from purely observational data. Continuing this line of research, we show how to discover the complete causal structure of continuous-valued data, under the assumptions that (a) the data generating process is linear, (b) there are no unobserved confounders, and (c) disturbance variables have non-Gaussian distributions of non-zero variances. The solution relies on the use of the statistical method known as independent component analysis, and does not require any pre-specified time-ordering of the variables. We provide a complete Matlab package for performing this LiNGAM analysis (short for Linear Non-Gaussian Acyclic Model), and demonstrate the effectiveness of the method using artificially generated data and real-world data.

We introduce two new functionals, the constrained covariance and the kernel mutual information, to measure the degree of independence of random variables. These quantities are both based on the covariance between functions of the random variables in reproducing kernel Hilbert spaces (RKHSs). We prove that when the RKHSs are universal, both functionals are zero if and only if the random variables are pairwise independent. We also show that the kernel mutual information is an upper bound near independence on the Parzen window estimate of the mutual information. Analogous results apply for two correlation-based dependence functionals introduced earlier: we show the kernel canonical correlation and the kernel generalised variance to be independence measures for universal kernels, and prove the latter to be an upper bound on the mutual information near independence. The performance of the kernel dependence functionals in measuring independence is verified in the context of independent component analysis.

This paper presents an unified approach to the problem of separation of sources, based on the consideration of mutual information. The basic setup is that the sources are independent stationary random processes which are mixed either instantaneously or through a convolution, to produce the observed records. We define the entropy of stationary processes and then the mutual information between them as a measure of their independence. This provides us with a contrast for the separation of source problem. For practical implementation, we introduce several degraded forms of this contrast, which can be computed from a finite dimensional distribution of the reconstructed source processes only. From them, we derive several sets of estimating equations generalising those considered earlier. 1 Introduction Blind separation of sources is a topic which have received much attention recently, as it has many important applications (speech analysis, radar, sonar, : : : ). Basically, one observes seve...