## Blind Source Separation and Deconvolution: The Dynamic Component Analysis Algorithm (1998)

Venue: | Neural Computation |

Citations: | 43 - 6 self |

### BibTeX

@ARTICLE{Attias98blindsource,

author = {H. Attias and C. E. Schreiner},

title = {Blind Source Separation and Deconvolution: The Dynamic Component Analysis Algorithm},

journal = {Neural Computation},

year = {1998},

volume = {10},

pages = {1373--1424}

}

### Years of Citing Articles

### OpenURL

### Abstract

We derive a novel family of unsupervised learning algorithms for blind separation of mixed and convolved sources. Our approach is based on formulating the separation problem as a learning task of a spatio-temporal generative model, whose parameters are adapted iteratively to minimize suitable error functions, thus ensuring stability of the algorithms. The resulting learning rules achieve separation by exploiting high-order spatio-temporal statistics of the mixture data. Different rules are obtained by learning generative models in the frequency and time domains, whereas a hybrid frequency/time model leads to the best performance. These algorithms generalize independent component analysis to the case of convolutive mixtures and exhibit superior performance on instantaneous mixtures. An extension of the relative-gradient concept to the spatio-temporal case leads to fast and efficient learning rules with equivariant properties. Our approach can incorporate information about the mixing sit...

### Citations

9359 |
Elements of information theory
- Cover, Thomas
- 1991
(Show Context)
Citation Context ... the joint distribution of Re(~u i;k ) and Im(~u i;k ). In Appendix A.1 we derive the model sensor density p ~ y (62) from (12,13). As our error function we choose the Kullback-Leibler (KL) distance (=-=Cover and Thomas 1991-=-) E(p o ~ y ; p ~ y ) (64), an asymmetric measure of the distance between the correct density p o ~ y and the model p ~ y . As shown in Appendix A.1, the DCA-IF (I=instantaneous mixing, F=frequency do... |

1468 |
Independent component analysis, a new concept
- Comon
- 1994
(Show Context)
Citation Context ...no delays or frequency distortion. A robust and efficient set of algorithms which perform `independent component analysis' (ICA) has been proven effective for a large class of instantaneous mixtures (=-=Comon 1994-=-; Bell and Sejnowski 1995; Cardoso and Laheld 1996; Pham 1996; Pearlmutter and Parra 1997). In that case we seek a separating transformation G ij that, when applied to the sensor signals y i (t) which... |

1191 | An information-maximization approach to blind separation and blind deconvolution
- Bell, Sejnowski
- 1995
(Show Context)
Citation Context ... frequency distortion. A robust and efficient set of algorithms which perform `independent component analysis' (ICA) has been proven effective for a large class of instantaneous mixtures (Comon 1994; =-=Bell and Sejnowski 1995-=-; Cardoso and Laheld 1996; Pham 1996; Pearlmutter and Parra 1997). In that case we seek a separating transformation G ij that, when applied to the sensor signals y i (t) which are generally correlated... |

562 |
Auditory Scene Analysis
- Bregman
- 1990
(Show Context)
Citation Context ...or noise. Currently, there exists no algorithm that can solve the general problem. The human auditory system, however, can solve it under some conditions for L 0 = 2 (the `cocktail party' effect; see =-=Bregman 1990-=-). Given the complexity of the actual problem, current work on blind separation focuses on an idealized version thereof where the mixing is square (L 0 = L), invertible, linear, noiseless, and time-in... |

541 | A New Learning Algorithm for Blind Signal Separation
- Amari, Cichocki, et al.
- 1996
(Show Context)
Citation Context ...tion for arbitrary source densities; (2) Stability is ensured by the error function optimization procedure; (3) Thanks to our generalization of the relative gradient concept (Cardoso and Laheld 1996; =-=Amari et al. 1996-=-) to the convolutive case, the learning is accelerated and possesses the property of equivariance, which guarantees uniform performance across the space of all invertible mixing processes, and thus ro... |

533 | Spikes: Exploring the neural code - Rieke, Warland, et al. - 1997 |

477 |
Blind separation of sources, Part I: an adaptive algorithm based on neuromimetic architecture, Signal Processing 24
- Jutten, Herault
- 1991
(Show Context)
Citation Context ... as an optimization problem using the KL distance, which is bounded from below (Cover and Thomas 1991), as an error function. Previously proposed separation methods, such as the original H-J network (=-=Jutten and Herault 1991-=-) and its extension to convolutive mixing by Platt and Faggin (1992), are not derived from an error function, except for special choices of the non-linearities (Comon, Jutten and Herault 1991; Sorouch... |

408 | Equivariant adaptive source separation
- Cardoso, Laheld
- 1996
(Show Context)
Citation Context ...robust and efficient set of algorithms which perform `independent component analysis' (ICA) has been proven effective for a large class of instantaneous mixtures (Comon 1994; Bell and Sejnowski 1995; =-=Cardoso and Laheld 1996-=-; Pham 1996; Pearlmutter and Parra 1997). In that case we seek a separating transformation G ij that, when applied to the sensor signals y i (t) which are generally correlated, will produce a new set ... |

255 |
Could information theory provide an ecological theory of sensory processing
- Atick
- 1992
(Show Context)
Citation Context ...89) suggestion of redundancy reduction as an important goal of sensory processing. This idea has been formulated as a quantitative theory for the subcortical visual system in (Atick and Redlich 1990; =-=Atick 1992-=-; Dong and Atick 1995). In (Bell and Sejnowski 1996), an attempt is made to construct a computational model of primary visual cortex cells based on ICA. Hopfield (1991) applied Jutten and Herault's (1... |

250 | Adaptive Filter Theory, 3rd ed - Haykin - 1996 |

244 | Innovation and Learning - Barlow |

234 | A blind source separation technique using second-order statistics - Belouchrani, Abed-Meraim, et al. - 1997 |

213 | Independent component analysis of electroencephalographic data
- Makeig, Bell, et al.
- 1996
(Show Context)
Citation Context ...m of blind source separation address, in fact, the more general need for an efficient tool for statistical analysis of spatio-temporal data sets, e.g., biomedical multi-sensor recordings such as EEG (=-=Makeig et al. 1996-=-, 1997) and MEG (Poeppel et al. 1997). The DCA separating filters produce simultaneous spatial and temporal redundancy reduction. Combined with dimensionality reduction as described in Section 7, the ... |

144 |
An Introduction to Latent Variable Models
- Everitt
- 1984
(Show Context)
Citation Context ...stantaneous DCA algorithms are more powerful than the spatial-based ICA ones. Second, we demonstrate that an effective way to impose spatio-temporal independence is via the use of a generative model (=-=Everitt 1984-=-) for the probability density of the sensor signals. Specifically, we construct a parametric model for the joint density of the L sensors over N-point time blocks, p y [y 1 (t 0 ); :::; y 1 (t N \Gamm... |

141 | Reading a neural code - Bialek, Rieke, et al. - 1991 |

135 | Blind separation of auditory event-related brain responses into independent components - Makeig, Jung, et al. - 1997 |

118 |
Towards a theory of early visual processing
- Atick, Redlich
- 1990
(Show Context)
Citation Context ...accord with Barlow's (1989) suggestion of redundancy reduction as an important goal of sensory processing. This idea has been formulated as a quantitative theory for the subcortical visual system in (=-=Atick and Redlich 1990-=-; Atick 1992; Dong and Atick 1995). In (Bell and Sejnowski 1996), an attempt is made to construct a computational model of primary visual cortex cells based on ICA. Hopfield (1991) applied Jutten and ... |

100 | Blind separation of convolved sources based on information maximization - Torkkola - 1996 |

83 | Infomax and maximum likelihood for source separation - Cardoso - 1997 |

81 | Multichannel blind deconvolution: FIR matrix algebra and separation of multipath mixtures
- Lambert
- 1995
(Show Context)
Citation Context ...CA-I. Torkkola (1996) proposed the ordinary-gradient rule corresponding to the frequency-domain version of (39), and its relative-gradient form was described by Lee, Bell and Lambert (1997) (see also =-=Lambert 1996-=-); in the absence of a spatio-temporal/spectral error function, both relied on information-maximization considerations in the frequency domain. A rule similar to (39) appeared in (Cochocki et al. 1996... |

79 | Maximum likelihood blind source separation: A context-sensitive generalization
- Pearlmutter, Parra
- 1997
(Show Context)
Citation Context ...ms which perform `independent component analysis' (ICA) has been proven effective for a large class of instantaneous mixtures (Comon 1994; Bell and Sejnowski 1995; Cardoso and Laheld 1996; Pham 1996; =-=Pearlmutter and Parra 1997-=-). In that case we seek a separating transformation G ij that, when applied to the sensor signals y i (t) which are generally correlated, will produce a new set of signalssx i (t),sx i (t) = L X j=1 G... |

71 | separation of instantaneous mixture of sources via an independent component analysis
- Pham, “Blind
- 1996
(Show Context)
Citation Context ...of algorithms which perform `independent component analysis' (ICA) has been proven effective for a large class of instantaneous mixtures (Comon 1994; Bell and Sejnowski 1995; Cardoso and Laheld 1996; =-=Pham 1996-=-; Pearlmutter and Parra 1997). In that case we seek a separating transformation G ij that, when applied to the sensor signals y i (t) which are generally correlated, will produce a new set of signalss... |

71 |
C.Jutten, “blind source separation for convolutive mixture
- Thi
- 1995
(Show Context)
Citation Context ...on considerations in the frequency domain. A rule similar to (39) appeared in (Cochocki et al. 1996). Methods that use cumulant information in the frequency domain (i.e., polyspectra) were suggested (=-=Thi and Jutten 1995-=-; Yellin and Weinstein 1995) but are restricted to L = 2 sources. Comon (1996) suggested a polyspectra-based optimization formulation but but did not provide a separation algorithm. DCA is sufficientl... |

69 | Blind separation of delayed and convolved sources - Lee, Bell, et al. - 1997 |

62 | Edges are the `independent components' of natural scenes
- Bell, Sejnowski
- 1997
(Show Context)
Citation Context ... as an important goal of sensory processing. This idea has been formulated as a quantitative theory for the subcortical visual system in (Atick and Redlich 1990; Atick 1992; Dong and Atick 1995). In (=-=Bell and Sejnowski 1996-=-), an attempt is made to construct a computational model of primary visual cortex cells based on ICA. Hopfield (1991) applied Jutten and Herault's (1991) ideas to explain odor discrimination. Furtherm... |

58 | Blind separation of sources - Comon, Jutten, et al. - 1991 |

47 | Temporal decorrelation: a theory of lagged and nonlagged responses in the lateral geniculate nucleus
- Dong, Atick
- 1995
(Show Context)
Citation Context ...on of redundancy reduction as an important goal of sensory processing. This idea has been formulated as a quantitative theory for the subcortical visual system in (Atick and Redlich 1990; Atick 1992; =-=Dong and Atick 1995-=-). In (Bell and Sejnowski 1996), an attempt is made to construct a computational model of primary visual cortex cells based on ICA. Hopfield (1991) applied Jutten and Herault's (1991) ideas to explain... |

46 | Contrast for multichannel blind deconvolution - Comon - 1996 |

39 | Separation of independent signals using timedelayed correlations - Molgedey, Schuster - 1994 |

28 |
separation of sources, Part III: Stability analysis
- Sorouchyari
- 1991
(Show Context)
Citation Context ...lt 1991) and its extension to convolutive mixing by Platt and Faggin (1992), are not derived from an error function, except for special choices of the non-linearities (Comon, Jutten and Herault 1991; =-=Sorouchyari 1991-=-; Comon 1994), and indeed exhibit occasional unstable behavior. Although the formulation of DCA in terms of learning a generative model assumes that the model approximates well the situation at hand, ... |

27 | Networks for the separation sources that are superimposed and delayed - Platt, Faggin - 1992 |

23 | Olfactory computation and object perception - Hopfield - 1991 |

11 | Coding of naturalistic stimuli by auditory midbrain neurons
- Attias, Schreiner
- 1998
(Show Context)
Citation Context ...hermore, the equivalent information-rate maximization of DCA given in Section 6 connects this approach to a recent line of physiological experiments (see, e.g., Bialek et al. 1991; Rieke et al. 1997; =-=Attias and Schreiner 1998-=-) that suggest that the nervous system is designed to maximize the rate at which spike trains carry information about the stimuli. The results of the present paper motivate us to hypothesize that the ... |

10 | Separation of Delayed and Convolved Signals with Self-Adaptive Learning - Cichocki, Amari, et al. - 1996 |

8 |
Low-order temporal statistics of natural sounds
- Attias, Schreiner
- 1997
(Show Context)
Citation Context ...e deriving the learning rules, let us make the whitened source model (13) more specific. First, we shall use the same parametrized functional form for all sources. This is consistent with our report (=-=Attias and Schreiner 1997-=-), which showed that a large class of natural sounds are characterized by the same parametric functional form of their frequency-domain density. Second, we shall omit the frequency-dependence of P i;k... |

1 |
Dynamic component analysis of auditory evoked neuromagnetic fields
- Poeppel, Attias, et al.
- 1997
(Show Context)
Citation Context ...s, in fact, the more general need for an efficient tool for statistical analysis of spatio-temporal data sets, e.g., biomedical multi-sensor recordings such as EEG (Makeig et al. 1996, 1997) and MEG (=-=Poeppel et al. 1997-=-). The DCA separating filters produce simultaneous spatial and temporal redundancy reduction. Combined with dimensionality reduction as described in Section 7, the resulting dynamic components of a gi... |