Abstract not found

A full-rank matrix A ∈ IR n×m with n < m generates an underdetermined system of linear equations Ax = b having infinitely many solutions. Suppose we seek the sparsest solution, i.e., the one with the fewest nonzero entries: can it ever be unique? If so, when? As optimization of sparsity is combinatorial in nature, are there efficient methods for finding the sparsest solution? These questions have been answered positively and constructively in recent years, exposing a wide variety of surprising phenomena; in particular, the existence of easily-verifiable conditions under which optimally-sparse solutions can be found by concrete, effective computational methods. Such theoretical results inspire a bold perspective on some important practical problems in signal and image processing. Several well-known signal and image processing problems can be cast as demanding solutions of undetermined systems of equations. Such problems have previously seemed, to many, intractable. There is considerable evidence that these problems often have sparse solutions. Hence, advances in finding sparse solutions to underdetermined systems energizes research on such signal and image processing problems – to striking effect. In this paper we review the theoretical results on sparse solutions of linear systems, empirical

Abstract—We address the problem of finding sparse solutions to an underdetermined system of equations when there are multiple measurement vectors having the same, but unknown, sparsity structure. The single measurement sparse solution problem has been extensively studied in the past. Although known to be NP-hard, many single–measurement suboptimal algorithms have been formulated that have found utility in many different applications. Here, we consider in depth the extension of two classes of algorithms–Matching Pursuit (MP) and FOCal Underdetermined System Solver (FOCUSS)–to the multiple measurement case so that they may be used in applications such as neuromagnetic imaging, where multiple measurement vectors are available, and solutions with a common sparsity structure must be computed. Cost functions appropriate to the multiple measurement problem are developed, and algorithms are derived based on their minimization. A simulation study is conducted on a test-case dictionary to show how the utilization of more than one measurement vector improves the performance of the MP and FOCUSS classes of algorithm, and their performances are compared. I.

This article extends the concept of compressed sensing to signals that are not sparse in an orthonormal basis but rather in a redundant dictionary. It is shown that a matrix, which is a composition of a random matrix of certain type and a deterministic dictionary, has small restricted isometry constants. Thus, signals that are sparse with respect to the dictionary can be recovered via Basis Pursuit from a small number of random measurements. Further, thresholding is investigated as recovery algorithm for compressed sensing and conditions are provided that guarantee reconstruction with high probability. The different schemes are compared by numerical experiments. Key words: compressed sensing, redundant dictionary, sparse approximation, random

Abstract not found

Abstract. This paper presents a framework for learning multiscale sparse representations of color images and video with overcomplete dictionaries. A single-scale K-SVD algorithm was introduced in [1], formulating sparse dictionary learning for grayscale image representation as an optimization problem, efficiently solved via Orthogonal Matching Pursuit (OMP) and Singular Value Decomposition (SVD). Following this work, we propose a multiscale learned representation, obtained by using an efficient quadtree decomposition of the learned dictionary, and overlapping image patches. The proposed framework provides an alternative to pre-defined dictionaries such as wavelets, and shown to lead to state-of-the-art results in a number of image and video enhancement and restoration applications. This paper describes the proposed framework, and accompanies it by numerous examples demonstrating its strength. Key words. Image and video processing, sparsity, dictionary, multiscale representation, denoising, inpainting, interpolation, learning. AMS subject classifications. 49M27, 62H35

The minimum ℓ1-norm solution to an underdetermined system of linear equations y = Ax, is often, remarkably, also the sparsest solution to that system. This sparsity-seeking property is of interest in signal processing and information transmission. However, general-purpose optimizers are much too slow for ℓ1 minimization in many large-scale applications. The Homotopy method was originally proposed by Osborne et al. for solving noisy overdetermined ℓ1-penalized least squares problems. We here apply it to solve the noiseless underdetermined ℓ1-minimization problem min ‖x‖1 subject to y = Ax. We show that Homotopy runs much more rapidly than general-purpose LP solvers when sufficient sparsity is present. Indeed, the method often has the following k-step solution property: if the underlying solution has only k nonzeros, the Homotopy method reaches that solution in only k iterative steps. When this property holds and k is small compared to the problem size, this means that ℓ1 minimization problems with k-sparse solutions can be solved in a fraction of the cost of solving one full-sized linear system. We demonstrate this k-step solution property for two kinds of problem suites. First,

This paper presents a new image representation method based on anisotropic refinement. It has been shown that wavelets are not optimal to code 2-D objects which need true 2-D dictionaries for efficient approximation. We propose to use rotations and anisotropic scaling to build a real bi-dimensional dictionary. Matching Pursuit then stands as a natural candidate to provide an image representation with an anisotropic refinement scheme. It basically decomposes the image as a series of basis functions weighted by their respective coefficients. Even if the basis functions can a priori take any form bi-dimensional dictionaries are almost exclusively composed of two-dimensional Gabor functions. We present here a new dictionary design by introducing orientation and anisotropic refinement of a gaussian generating function. The new dictionary permits to efficiently code 2-D objects and more particularly oriented contours. It is shown to clearly outperform common non-oriented Gabor dictionaries.

This paper proposes a rate-distortion optimal a posteriori quantization scheme for Matching Pursuit coefficients. The a posteriori quantization applies to a Matching Pursuit expansion that has been generated off-line, and cannot benefit of any feedback loop to the encoder in order to compensate for the quantization noise. The redundancy of the Matching Pursuit dictionary provides an indicator of the relative importance of coefficients and atom indices, and subsequently on the quantization error. It is used to define a universal upper-bound on the decay of the coefficients, sorted in decreasing order of magnitude. A new quantization scheme is then derived, where this bound is used as an Oracle for the design of an optimal a posteriori quantizer. The latter turns the exponentially distributed coefficient entropy-constrained quantization problem into a simple uniform quantization problem. Using simulations

this paper is to provide methods that efficiently produce approximate solutions of very large systems of the above form. In addition, we concentrate on approximate solutions with only few nonzero coefficients ff j (f; X). The reason is that the evaluation of a full sum in (1) on many points will be too costly, if the sum contains a term for each data value, and if sparsity is limited for reasons of approximation quality. In short, we try to approximate N data with K !! N terms, and we want to keep the storage and computational effort proportional to N . This implies that we try to avoid storage of the full matrix AX .