## A Context-Sensitive Generalization of ICA (1996)

### Cached

### Download Links

Citations: | 96 - 8 self |

### BibTeX

@MISC{Pearlmutter96acontext-sensitive,

author = {Barak Pearlmutter and Lucas Parra},

title = {A Context-Sensitive Generalization of ICA},

year = {1996}

}

### Years of Citing Articles

### OpenURL

### Abstract

Source separation arises in a surprising number of signal processing applications, from speech recognition to EEG analysis. In the square linear blind source separation problem without time delays, one must find an unmixing matrix which can detangle the result of mixing n unknown independent sources through an unknown n \Theta n mixing matrix. The recently introduced ICA blind source separation algorithm (Baram and Roth 1994; Bell and Sejnowski 1995) is a powerful and surprisingly simple technique for solving this problem. ICA is all the more remarkable for performing so well despite making absolutely no use of the temporal structure of its input! This paper presents a new algorithm, contextual ICA, which derives from a maximum likelihood density estimation formulation of the problem. cICA can incorporate arbitrarily complex adaptive history-sensitive source models, and thereby make use of the temporal structure of its input. This allows it to separate in a number of situations where s...

### Citations

1367 | Independent Component Analysis: a new concept - Comon - 1994 |

1085 | An Information Approach to Blind Separation
- Bell, Sejnowski
- 1995
(Show Context)
Citation Context ...atrix which can detangle the result of � mixing unknown independent sources through an ���¤� unknown mixing matrix. The recently introduced ICA blind source separation algorithm (Baram and Roth 1994; =-=Bell and Sejnowski 1995-=-) is a powerful and surprisingly simple technique for solving this problem. ICA is all the more remarkable for performing so well despite making absolutely no use of the temporal structure of its inpu... |

564 |
A stochastic approximation method
- Robbins, Monro
- 1951
(Show Context)
Citation Context ...it can be obtained by taking a sample � from � , �� ��� ������������� ����������� (2) � In order to apply a stochastic gradient optimization method, we wish to find an unbiased ��� � � estimate of � (=-=Robbins and Monro 1951-=-). Due to the linearity � ����������� of differentiation, is such an estimate. For blind source separation, we consider the parametric form � ����������� for shown in figure 1. � Let be � an - dimensi... |

528 |
Analysis of a complex of statistical variables into principal components
- Hotelling
- 1933
(Show Context)
Citation Context ...he history used is varied from zero, which corresponds to conventional ICA (left), to one (center), to two (right). Finally, we would like to compare ICA with PCA. The principal components algorithm (=-=Hotelling 1933-=-) fits a linear mixture of one-dimensional Gaussian sources of minimal variance to samples from a high-dimensional distribution. ICA performs a similar action, but instead uses a linear mixture of pot... |

514 | A new learning algorithm for blind signal separation - Amari, Cichocki, et al. - 1996 |

194 | Independent component analysis of electroencephalographic data
- Makeig, Bell, et al.
- 1996
(Show Context)
Citation Context ...tions, has been applied to a surprising number of problems, from separation of digitally mixed speech signals (Bell and Sejnowski 1995), to separating the componenets of electroencephalographic data (=-=Makeig et al. 1996-=-), to blind deconvolution (Bell and Sejnowski 1995), to finding the higher-order structure of a natural sound (Bell and Sejnowski 1996b), and even to financial forecasting (Baram and Roth 1995) and im... |

124 | Reading a neural code - Bialek, Rieke, et al. - 1991 |

94 | Blind separation of convolved sources based on information maximisation - Torkkola - 1996 |

65 | Learning the higher-order structure of a natural sound
- Bell, Sejnowski
- 1996
(Show Context)
Citation Context ... 1995), to separating the componenets of electroencephalographic data (Makeig et al. 1996), to blind deconvolution (Bell and Sejnowski 1995), to finding the higher-order structure of a natural sound (=-=Bell and Sejnowski 1996-=-b), and even to financial forecasting (Baram and Roth 1995) and image processing (Bell and Sejnowski 1996a). There have been attempts to generalize the algorithm, the most notable being extensions to ... |

33 | separation of delayed sources based on information maximization - Torkkola, “Blind |

21 |
Maximum-Likelihood Deconvolution: A Journey into Model-Based Signal Processing
- Mendel, Burrus
- 1990
(Show Context)
Citation Context ...blem of density estimation from samples. One desires to estimate some true distribu������� tion over a �¤� space from which ���������¨������� samples have been drawn. The maximum likelihood approach (=-=Mendel and Burrus 1990-=-) is to use a density estimator of some parametric form, � ����������� say . Given a setting of the parameter � vector , this will constitute the estimated probability density. In order to � set appro... |

11 | Adaptive soft weight tying using gaussian mixtures - Nowlan, Hinton - 1991 |

9 | Redundancy reduction with informa tion-preserving maps - Parra, Deco, et al. - 1995 |

7 |
Density shaping by neural networks with application to classification, estimation and forecasting (Tech. Rep
- Baram, Roth
- 1994
(Show Context)
Citation Context ...st find an unmixing matrix which can detangle the result of � mixing unknown independent sources through an ���¤� unknown mixing matrix. The recently introduced ICA blind source separation algorithm (=-=Baram and Roth 1994-=-; Bell and Sejnowski 1995) is a powerful and surprisingly simple technique for solving this problem. ICA is all the more remarkable for performing so well despite making absolutely no use of the tempo... |

6 |
Forecasting by Density Shaping Using Neural Networks
- Baram, Roth
- 1995
(Show Context)
Citation Context ...ic data (Makeig et al. 1996), to blind deconvolution (Bell and Sejnowski 1995), to finding the higher-order structure of a natural sound (Bell and Sejnowski 1996b), and even to financial forecasting (=-=Baram and Roth 1995-=-) and image processing (Bell and Sejnowski 1996a). There have been attempts to generalize the algorithm, the most notable being extensions to tolerate time delays and echos introduced by Torkkola (199... |

4 | Nouveaux Memoires de l'Academie Royale des Sciences, des Lettres et des Beaux–Arts de Belgique 18 - Verhulst |

3 | Temporally Continuous vs. Clocked Networks - Pearlmutter - 1992 |

2 | Linear Feature Extraction in non-Gaussian Networks - Obradovic, Deco - 1995 |

1 | Stochastic gradient descent In the above experiments a number of techniques were used to improve the efficiency and robustness of the stochastic gradient descent procedure as applied to cICA. First, rather than performing gradient descent directly on the - unknown authors - 1992 |