Results 1  10
of
148
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
, 2007
"... A fullrank matrix A ∈ IR n×m with n < m generates an underdetermined system of linear equations Ax = b having infinitely many solutions. Suppose we seek the sparsest solution, i.e., the one with the fewest nonzero entries: can it ever be unique? If so, when? As optimization of sparsity is combinato ..."
Abstract

Cited by 202 (31 self)
 Add to MetaCart
A fullrank matrix A ∈ IR n×m with n < m generates an underdetermined system of linear equations Ax = b having infinitely many solutions. Suppose we seek the sparsest solution, i.e., the one with the fewest nonzero entries: can it ever be unique? If so, when? As optimization of sparsity is combinatorial in nature, are there efficient methods for finding the sparsest solution? These questions have been answered positively and constructively in recent years, exposing a wide variety of surprising phenomena; in particular, the existence of easilyverifiable conditions under which optimallysparse solutions can be found by concrete, effective computational methods. Such theoretical results inspire a bold perspective on some important practical problems in signal and image processing. Several wellknown signal and image processing problems can be cast as demanding solutions of undetermined systems of equations. Such problems have previously seemed, to many, intractable. There is considerable evidence that these problems often have sparse solutions. Hence, advances in finding sparse solutions to underdetermined systems energizes research on such signal and image processing problems – to striking effect. In this paper we review the theoretical results on sparse solutions of linear systems, empirical
S.: Blind separation of speech mixtures via timefrequency masking
 IEEE Transactions on Signal Processing (2002) Submitted
"... Abstract—Binary timefrequency masks are powerful tools for the separation of sources from a single mixture. Perfect demixing via binary timefrequency masks is possible provided the timefrequency representations of the sources do not overlap: a condition we calldisjoint orthogonality. We introduc ..."
Abstract

Cited by 197 (3 self)
 Add to MetaCart
Abstract—Binary timefrequency masks are powerful tools for the separation of sources from a single mixture. Perfect demixing via binary timefrequency masks is possible provided the timefrequency representations of the sources do not overlap: a condition we calldisjoint orthogonality. We introduce here the concept of approximatedisjoint orthogonality and present experimental results demonstrating the level of approximate Wdisjoint orthogonality of speech in mixtures of various orders. The results demonstrate that there exist ideal binary timefrequency masks that can separate several speech signals from one mixture. While determining these masks blindly from just one mixture is an open problem, we show that we can approximate the ideal masks in the case where two anechoic mixtures are provided. Motivated by the maximum likelihood mixing parameter estimators, we define a power weighted twodimensional (2D) histogram constructed from the ratio of the timefrequency representations of the mixtures that is shown to have one peak for each source with peak location corresponding to the relative attenuation and delay mixing parameters. The histogram is used to create timefrequency masks that partition one of the mixtures into the original sources. Experimental results on speech mixtures verify the technique. Example demixing results can be found online at
Image Decomposition via the Combination of Sparse Representations and a Variational Approach
 IEEE Transactions on Image Processing
, 2004
"... The separation of image content into semantic parts plays a vital role in applications such as compression, enhancement, restoration, and more. In recent years several pioneering works suggested such a separation based on variational formulation, and others using independent component analysis and s ..."
Abstract

Cited by 127 (27 self)
 Add to MetaCart
The separation of image content into semantic parts plays a vital role in applications such as compression, enhancement, restoration, and more. In recent years several pioneering works suggested such a separation based on variational formulation, and others using independent component analysis and sparsity. This paper presents a novel method for separating images into texture and piecewise smooth (cartoon) parts, exploiting both the variational and the sparsity mechanisms. The method combines the Basis Pursuit Denoising (BPDN) algorithm and the TotalVariation (TV) regularization scheme. The basic idea presented in this paper is the use of two appropriate dictionaries, one for the representation of textures, and the other for the natural scene parts, assumed to be piecewisesmooth. Both dictionaries are chosen such that they lead to sparse representations over one type of imagecontent (either texture or piecewise smooth). The use of the BPDN with the two augmented dictionaries leads to the desired separation, along with noise removal as a byproduct. As the need to choose proper dictionaries is generally hard, a TV regularization is employed to better direct the separation process and reduce ringing artifacts. We present a highly e#cient numerical scheme to solve the combined optimization problem posed in our model, and show several experimental results that validate the algorithm's performance.
Online learning for matrix factorization and sparse coding
"... Sparse coding—that is, modelling data vectors as sparse linear combinations of basis elements—is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on the largescale matrix factorization problem that consists of learning the basis set, adapting it t ..."
Abstract

Cited by 97 (18 self)
 Add to MetaCart
Sparse coding—that is, modelling data vectors as sparse linear combinations of basis elements—is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on the largescale matrix factorization problem that consists of learning the basis set, adapting it to specific data. Variations of this problem include dictionary learning in signal processing, nonnegative matrix factorization and sparse principal component analysis. In this paper, we propose to address these tasks with a new online optimization algorithm, based on stochastic approximations, which scales up gracefully to large datasets with millions of training samples, and extends naturally to various matrix factorization formulations, making it suitable for a wide range of learning problems. A proof of convergence is presented, along with experiments with natural images and genomic data demonstrating that it leads to stateoftheart performance in terms of speed and optimization for both small and large datasets.
Highly sparse representations from dictionaries are unique and independent of the sparseness measure
, 2003
"... ..."
Survey of Sparse and NonSparse Methods in Source Separation
, 2005
"... Source separation arises in a variety of signal processing applications, ranging from speech processing to medical image analysis. The separation of a superposition of multiple signals is accomplished by taking into account the structure of the mixing process and by making assumptions about the sour ..."
Abstract

Cited by 35 (1 self)
 Add to MetaCart
Source separation arises in a variety of signal processing applications, ranging from speech processing to medical image analysis. The separation of a superposition of multiple signals is accomplished by taking into account the structure of the mixing process and by making assumptions about the sources. When the information about the mixing process and sources is limited, the problem is called ‘blind’. By assuming that the sources can be represented sparsely in a given basis, recent research has demonstrated that solutions to previously problematic blind source separation problems can be obtained. In some cases, solutions are possible to problems intractable by previous nonsparse methods. Indeed, sparse methods provide a powerful approach to the separation of linear mixtures of independent data. This paper surveys the recent arrival of sparse blind source separation methods and the previously existing nonsparse methods, providing insights and appropriate hooks into the literature along the way.
Proposals for performance measurement in source separation
 in Proc. 4th Int. Symp. on Independent Component Anal. and Blind Signal Separation (ICA2003
, 2003
"... In this paper, we address a few issues related to the evaluation of the performance of source separation algorithms. We propose several measures of distortion that take into account the gain indeterminacies of BSS algorithms. The total distortion includes interference from the other sources as well ..."
Abstract

Cited by 31 (12 self)
 Add to MetaCart
In this paper, we address a few issues related to the evaluation of the performance of source separation algorithms. We propose several measures of distortion that take into account the gain indeterminacies of BSS algorithms. The total distortion includes interference from the other sources as well as noise and algorithmic artifacts, and we define performance criteria that measure separately these contributions. The criteria are valid even in the case of correlated sources. When the sources are estimated from a degenerate set of mixtures by applying a demixing matrix, we prove that there are upper bounds on the achievable Source to Interference Ratio. We propose these bounds as benchmarks to assess how well a (linear or nonlinear) BSS algorithm performs on a set of degenerate mixtures. We demonstrate on an example how to use these figures of merit to evaluate and compare the performance of BSS algorithms. 1.
A simple test to check the optimality of sparse signal approximations
 Tech. Rep., IRISA, 2004, in preparation. V  720
"... Approximating a signal or an image with a sparse linear expansion from an overcomplete dictionary of atoms is an extremely useful tool to solve many signal processing problems. Finding the sparsest approximation of a signal from an arbitrary dictionary is an NPhard problem. Despite of this, several ..."
Abstract

Cited by 30 (2 self)
 Add to MetaCart
Approximating a signal or an image with a sparse linear expansion from an overcomplete dictionary of atoms is an extremely useful tool to solve many signal processing problems. Finding the sparsest approximation of a signal from an arbitrary dictionary is an NPhard problem. Despite of this, several algorithms have been proposed that provide suboptimal solutions. However, it is generally difficult to know how close the computed solution is to being “optimal”, and whether another algorithm could provide a better result. In this paper we provide a simple test to check whether the output of a sparse approximation algorithm is nearly optimal, in the sense that no significantly different linear expansion from the dictionary can provide both a smaller approximation error and a better sparsity. As a byproduct of our theorems, we obtain results on the identifiability of sparse overcomplete models in the presence of noise, for a fairly large class of sparse priors. 1.
Underdetermined blind source separation based on sparse representation
 IEEE Transactions on Signal Processing
, 2006
"... Abstract—This paper discusses underdetermined (i.e., with more sources than sensors) blind source separation (BSS) using a twostage sparse representation approach. The first challenging task of this approach is to estimate precisely the unknown mixing matrix. In this paper, an algorithm for estimat ..."
Abstract

Cited by 28 (8 self)
 Add to MetaCart
Abstract—This paper discusses underdetermined (i.e., with more sources than sensors) blind source separation (BSS) using a twostage sparse representation approach. The first challenging task of this approach is to estimate precisely the unknown mixing matrix. In this paper, an algorithm for estimating the mixing matrix that can be viewed as an extension of the DUET and the TIFROM methods is first developed. Standard clustering algorithms (e.g., Kmeans method) also can be used for estimating the mixing matrix if the sources are sufficiently sparse. Compared with the DUET, the TIFROM methods, and standard clustering algorithms, with the authors ’ proposed method, a broader class of problems can be solved, because the required key condition on sparsity of the sources can be considerably relaxed. The second task of the twostage approach is to estimate the source matrix using a standard linear programming algorithm. Another main contribution of the work described in this paper is the development of a recoverability analysis. After extending the results in [7], a necessary and sufficient condition for recoverability of a source vector is obtained. Based on this condition and various types of source sparsity, several probability inequalities and probability estimates for the recoverability issue are established. Finally, simulation results that illustrate the effectiveness of the theoretical results are presented. Index Terms—Blind source separation (BSS), Inorm, probability, recoverability, sparse representation, wavelet packets. I.