## Survey of Sparse and Non-Sparse Methods in Source Separation (2005)

### Cached

### Download Links

Citations: | 36 - 1 self |

### BibTeX

@MISC{O’Grady05surveyof,

author = {Paul D. O’Grady and Barak A. Pearlmutter and Scott T. Rickard},

title = {Survey of Sparse and Non-Sparse Methods in Source Separation},

year = {2005}

}

### OpenURL

### Abstract

Source separation arises in a variety of signal processing applications, ranging from speech processing to medical image analysis. The separation of a superposition of multiple signals is accomplished by taking into account the structure of the mixing process and by making assumptions about the sources. When the information about the mixing process and sources is limited, the problem is called ‘blind’. By assuming that the sources can be represented sparsely in a given basis, recent research has demonstrated that solutions to previously problematic blind source separation problems can be obtained. In some cases, solutions are possible to problems intractable by previous non-sparse methods. Indeed, sparse methods provide a powerful approach to the separation of linear mixtures of independent data. This paper surveys the recent arrival of sparse blind source separation methods and the previously existing non-sparse methods, providing insights and appropriate hooks into the literature along the way.

### Citations

8090 | Maximum likelihood from incomplete data via the EM algorithm
- Dempster, Laird, et al.
- 1977
(Show Context)
Citation Context ... gradient algorithm is replaced by an eigenvector decomposition of the covariance matrix of the data set (O’Grady and Pearlmutter, 2004b). This approach is an expectation maximisation (EM) algorithm (=-=Dempster et al., 1976-=-), where the E-step calculates posterior probabilities assigning data points to lines and M-step repositions the lines to match the points assigned to them. The separation of 6 sources from 4 mixtures... |

1923 | D: Pattern Classification - Duda, Hart, et al. - 2000 |

1857 | Some Methods for classification and Analysis of Multivariate Observations - MacQueen - 1967 |

1652 | Atomic decomposition by basis pursuit
- Chen, Donoho, et al.
- 2001
(Show Context)
Citation Context ...ent of each data point to multiple sources. This is generally done by minimisation of the L1-norm (sometimes referred to as the shortest-path algorithm (Bofill and Zibulevsky, 2000) or basis pursuit (=-=Chen et al., 1998-=-)). L1-norm minimisation is a piecewise linear operation that partially assigns the energy of x(t) to the M columns of Â that form a cone around x(t)inR M space. The remaining N – M columns are assign... |

1493 | Topographic independent component analysis - Hyvarinen, Hoyer, et al. |

1357 |
Independent component analysis, a new concept
- Comon
- 1994
(Show Context)
Citation Context ... reshaping Linear filtering Herault and Jutten (1986) Independence maximisation Adaptive feedback network JADE (Cardoso and Joint cumulant diagonalisation Linear transformation Souloumiac, 1993) ICA (=-=Comon, 1994-=-) Mutual information minimisation Linear transformation BS-InfoMax (Bell and Entropy maximisation Linear transformation Sejnowski, 1995) Lambert (1995) Multichannel blind deconvolution Current estimat... |

1070 | An Information-Maximization Approach to Blind Separation and Blind Deconvolution - Bell, Sejnowski - 1995 |

851 |
Multirate Systems and Filter Banks
- Vaidyanathan
- 1993
(Show Context)
Citation Context ...hod) at the output of that stage. The number of stages is not specified in advance. Instead, the algorithm continues until no further improvement can be achieved, which differs from previous methods (=-=Vaidyanathan, 1993-=-) of paraunitary matrix estimation. Monaural source separation of convolutive sources has also received some attention. A biologically inspired technique that exploits spectral cues is presented in Pe... |

734 | H.S.: Algorithms for non-negative matrix factorization - Lee, Seung - 2001 |

510 | A New Learning Algorithm for Blind Signal Separation - Amari, Cichocki, et al. - 1996 |

491 | Blind beamforming for non-gaussian signals
- Cardoso, Souloumiac
- 1993
(Show Context)
Citation Context ...) Filter Power spectrum reshaping Linear filtering Herault and Jutten (1986) Independence maximisation Adaptive feedback network JADE (Cardoso and Joint cumulant diagonalisation Linear transformation =-=Souloumiac, 1993-=-) ICA (Comon, 1994) Mutual information minimisation Linear transformation BS-InfoMax (Bell and Entropy maximisation Linear transformation Sejnowski, 1995) Lambert (1995) Multichannel blind deconvoluti... |

463 |
Auditory Scene Analysis, The perceptual organization of sound
- Bregman
- 1990
(Show Context)
Citation Context ... application of NSC for monaural sound separation using a temporal continuity assumption is presented in Virtanen (2003). The Temporal continuity assumption is motivated by the human auditory system (=-=Bregman, 1990-=-) and assumes that the mixture sources have constant spectra with time-varying gain. This algorithm considers only monaural signals; therefore, a different generative model is used. The observed signa... |

433 | Blind separation of sources, part I: an adaptive algorithm based on neuromimetic architecture, Signal Process - Jutten, Herault - 1991 |

429 | Oja E: A fast fixed-point algorithm for independent component analysis. Neural Computation
- Hyvärinen
- 1997
(Show Context)
Citation Context ...ultichannel blind deconvolution Current estimate feedback Lin et al. (1997) a Hough transform Hard assignment SOBI (Belouchrani et al., 1997) Joint unitary diagonalisation Linear transformation fICA (=-=Hyvärinen and Oja, 1997-=-) Non-Gaussianity maximisation Linear transformation Lee et al. (1999) a Gradient ascent learning Maximum a posteriori DUET (Jourjine et al., 2000) a 2D Histogram clustering Binary time-frequency mask... |

389 | signal separation: Statistical principles - Cardoso, “Blind - 1998 |

377 | Extrapolation, Interpolation, and Smoothing of Stationary Time Series - Wiener - 1949 |

371 |
On lines and planes of closest fit to systems of points in space
- Pearson
- 1902
(Show Context)
Citation Context ... the resultant principal eigenvector i.e. the eigenvector with the largest eigenvalue, indicates the direction of the data. This calculation forms the basis of the principal component analysis (PCA) (=-=Pearson, 1901-=-) dimensionality reduction technique, which is based on second order statistics, and is considered here in a degenerate form in which only one principal component is retained. In the case of identifyi... |

257 | Learning overcomplete representations - Lewicki, Sejnowski |

247 |
Some experiments on the recognition of speech with one and with two ears
- Cherry
- 1953
(Show Context)
Citation Context ... assumptions about the surroundings in which the sensor observations are made also influence the complexity of the problem. BSS of acoustic signals is often referred to as the cocktail party problem (=-=Cherry, 1953-=-); that is the separation of individual voices from a myriad of voices in an uncontrolled acoustic environment such as a cocktail party. Sensor observations in a natural environment are confounded by ... |

227 | Pattern Classification, 2nd edition - Duda, Hart, et al. - 2008 |

199 | Blind separation of speech mixtures via time-frequency masking - Yilmaz, Rickard |

191 | Blind source separation by sparse decomposition - Zibulevsky, Pearlmutter, et al. - 2001 |

162 | Analysis of fMRI data by blind separation into independent spatial components
- McKeown, Makeig, et al.
- 1998
(Show Context)
Citation Context ...d to acoustic signals. BSS has also been applied to the decomposition of functional brain imaging data such as electroencephalography (Jung et al., 1999, 2000), functional magnetic resonance imaging (=-=McKeown et al., 1998-=-), and magnetoencephalography (Tang et al., 2000; Vigário et al., 2000; Wübbeler et al., 2000; Ziehe et al., 2000; Pearlmutter and Jaramillo, 2003). BSS has also been applied to such diverse areas as ... |

147 | separation of convolved mixtures in the frequency domain - Smaragdis, “Blind - 1998 |

142 | Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications - Cichocki, Amari - 2002 |

133 | Infomax and maximum likelihood for blind source separation - Cardoso - 2002 |

124 | Removing electroencephalographic artifacts by blind source separation, Psychophysiology 37 - Jung, Makeig, et al. - 2000 |

119 | Separation of a mixture of independent sources through a maximum likelihood approach - Pham, Garrat, et al. - 1992 |

109 | Underdetermined blind source separation using sparse representations - Bofill, Zibulevsky - 2001 |

107 | One microphone source separation - Roweis - 2000 |

105 | Non-negative sparse coding
- Hoyer
- 2002
(Show Context)
Citation Context ...ation (number of rows of S) equal to 10, starting from random initial values. NMF (6): features learned by NMF, with dimensionality 6. NMF (10): features learned by NMF, with dimensionality 10. (From =-=Hoyer, 2002-=-, Figure 1, with permission). ij Vol. 15, 18–33 (2005) 29sreplaced by a projected gradient descent calculation with the resultant update elements set to zero if negative and each column normalised. An... |

103 |
Adaptive noise cancelling: Principles and applications
- Widrow, Glover, et al.
- 1975
(Show Context)
Citation Context ...fications that allow existing instantaneous case algorithms to be used for anechoic situations have been proposed. Platt and Faggin (1992) apply the mathematics of adaptive noise cancelling networks (=-=Widrow et al., 1975-=-) to the source separation network of Herault and Jutten (1986). The adaption of noise cancellation networks is based on an elegant notion: if a signal is corrupted by noise, it will have higher power... |

100 | Blind source separation of more sources than mixtures using overcomplete representations - LEE, LEWICKI, et al. - 1999 |

98 | When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts - Donoho, Stodden - 2004 |

96 |
A blind source separation technique based on second order statistics
- Belouchrani, Meraim, et al.
- 1997
(Show Context)
Citation Context ...ell and Entropy maximisation Linear transformation Sejnowski, 1995) Lambert (1995) Multichannel blind deconvolution Current estimate feedback Lin et al. (1997) a Hough transform Hard assignment SOBI (=-=Belouchrani et al., 1997-=-) Joint unitary diagonalisation Linear transformation fICA (Hyvärinen and Oja, 1997) Non-Gaussianity maximisation Linear transformation Lee et al. (1999) a Gradient ascent learning Maximum a posterior... |

95 | A contex-sensitive generalization of ica - Pearlmutter, Parra - 1996 |

93 | separation of convolved sources based on information maximization - Blind - 1996 |

88 | An information-theoretic analysis of hard and soft assignment methods for clustering - Kearns, Mansour, et al. - 1997 |

76 | A Ziehe, An approach to blind source separation based on temporal structure of speech signals, Neurocomputing - Murata, Ikeda - 2001 |

74 | Blind separation of disjoint orthogonal signals: Demixing n sources from 2 mixtures
- Jourjine, Rickard, et al.
- 2000
(Show Context)
Citation Context ...unitary diagonalisation Linear transformation fICA (Hyvärinen and Oja, 1997) Non-Gaussianity maximisation Linear transformation Lee et al. (1999) a Gradient ascent learning Maximum a posteriori DUET (=-=Jourjine et al., 2000-=-) a 2D Histogram clustering Binary time-frequency masking Bofill and Zibulevsky (2000) a Potential function clustering L1-Norm minimisation Zibulevsky and Pearlmutter (2001) a MAP with Laplacian prior... |

73 | Multichannel blind deconvolution: FIR matrix algebra and separation of multipath mixtures
- Lambert
- 1996
(Show Context)
Citation Context ... to transform an FIR filter matrix to an FIR polynomial matrix by performing a frequency transform on its elements. FIR polynomial matrices are matrices whose elements are complex valued polynomials (=-=Lambert, 1996-=-). A consequence of the transformation is that the frequency components observed at the sensors are in fact instantaneous mixtures of the original frequency components of the sources, i.e each frequen... |

68 | Space or time adaptive signal processing by neural network models - Herault, Jutten - 1986 |

67 | Non-negative matrix factor deconvolution; extraction of multiple sound sources from monophonic inputs - Smaragdis - 2004 |

64 | Conditions for Nonnegative Independent Component Analysis - Plumbley |

60 |
Learning the parts of objects with nonnegative matrix factorization
- Lee, Seung
- 1999
(Show Context)
Citation Context ...any invertible matrix. However, we are now in a position to formulate X AS as an optimisation problem. We will focus our discussion on the emerging field of non-negative matrix factorisation (NMF) (=-=Lee and Seung, 1999-=-), which is an optimisation problem of the following form minimize CðX; A; SÞ subject to A; S 0: The presented cost function is convex in A or S only, but not in both variables together. Therefore, it... |

60 | An application of the principle of maximum information preservation to linear systems - Linsker - 1989 |

59 | A first application of independent component analysis to extracting structure from stock returns
- Back, Weigend
- 1997
(Show Context)
Citation Context ... has also been applied to such diverse areas as real time robot audition (Nakadai et al., 2002), digital watermark attacks (Du et al., 2002), and financial time series analysis (Roth and Baram, 1996; =-=Back and Weigend, 1997-=-). It has even been conjectured that BSS will have a role in the analysis of the Cosmic Microwave Background (Cardoso et al., 2003), potentially helping to elucidate the very origins of the universe. ... |

57 | Independent component approach to the analysis of EEG and MEG recordings - Vigário, Särelä, et al. |

54 | Independent Component Analysis: Principles and Practice - Roberts, Everson - 2001 |

50 | Sparse coding in the primate cortex - Foldiak - 2003 |