The goal of sparse approximation problems is to represent a target signal approximately as a linear combination of a few elementary signals drawn from a fixed collection. This paper surveys the major practical algorithms for sparse approximation. Specific attention is paid to computational issues, to the circumstances in which individual methods tend to perform well, and to the theoretical guarantees available. Many fundamental questions in electrical engineering, statistics, and applied mathematics can be posed as sparse approximation problems, making these algorithms versatile and relevant to a wealth of applications.

The variational approach to signal restoration calls for the minimization of a cost function that is the sum of a data fidelity term and a regularization term, the latter term constituting a ‘prior’. A synthesis prior represents the sought signal as a weighted sum of ‘atoms’. On the other hand, an analysis prior models the coefficients obtained by applying the forward transform to the signal. For orthonormal transforms, the synthesis prior and analysis prior are equivalent; however, for overcomplete transforms the two formulations are different. We compare analysis and synthesis ℓ1-norm regularization with overcomplete transforms for denoising and deconvolution.

This paper addresses the regularization by sparsity constraints by means of weighted ℓ p penalties for 0 ≤ p ≤ 2. For 1 ≤ p ≤ 2 special attention is payed to convergence rates in norm and to source conditions. As main results it is proven that one gets a convergence rate of √ δ in the 2-norm for 1 < p ≤ 2 and in the 1-norm for p = 1 as soon as the unknown solution is sparse. The case p = 1 needs a special technique where not only Bregman distances but also a so-called Bregman-Taylor distance has to be employed. For p < 1 only preliminary results are shown. These results indicate that, different from p ≥ 1, the regularizing properties depend on the interplay of the operator and the basis of sparsity. A counterexample for p = 0 shows that regularization need not to happen. AMS Subject classification: Primary 47A52; Secondary 65J20, 65F22. 1

In convex optimization, duality theory can sometimes lead to simpler solution methods than those resulting from direct primal analysis. In this paper, this principle is applied to a class of composite variational problems arising in particular in signal recovery. These problems are not easily amenable to solution by current methods but they feature Fenchel-Moreau-Rockafellar dual problems that can be solved by forward-backward splitting. The proposed algorithm produces simultaneously a sequence converging weakly to a dual solution, and a sequence converging strongly to the primal solution. Our framework is shown to capture and extend several existing duality-based signal recovery methods and to be applicable to a variety of new problems beyond their scope.

Preprint 10The consecutive numbering of the publications is determined by their chronological order. The aim of this preprint series is to make new research rapidly available for scientific discussion. Therefore, the responsibility for the contents is solely due to the authors. The publications will be distributed by the authors. Minimization of non-smooth, non-convex functionals by iterative thresholding

Abstract not found

A new theory named compressed sensing for simultaneous sampling and compression of signals has been becoming popular in the communities of signal processing, imaging and applied mathematics. In this paper, we present improved/accelerated iterative curvelet thresholding methods for compressed sensing reconstruction in the fields of remote sensing. Some recent strategies including Bioucas-Dias and Figueiredo’s two-step iteration, Beck and Teboulle’s fast method, and Osher et al’s linearized Bregman iteration are applied to iterative curvelet thresholding in order to accelerate convergence. Advantages and disadvantages of the proposed methods are studied using the so-called pseudo-Pareto curve in the numerical experiments on single-pixel remote sensing and Fourier-domain random imaging.

Preprint 11The consecutive numbering of the publications is determined by their chronological order. The aim of this preprint series is to make new research rapidly available for scientific discussion. Therefore, the responsibility for the contents is solely due to the authors. The publications will be distributed by the authors. Regularization with non-convex separable constraints

Several problems in signal and image processing involve non-smooth convex optimization problems. Popular examples are T V methods in image processing or decoding by ℓ 1 minimiziation in compressed sensing and regularization of inverse problems [3]. In this talk we study first order methods for the minimization of functionals of the form S + R where S is differentiable with Lipschitz continuous derivative and R is convex. In particular we deal with forward-backward splitting methods of the form u n+1 = (I + ∂R) −1 (u n − snS ′ (u n)) which exploit that the operator (I + ∂R) may be easily invertible [2]. We derive a resembling method as a generalized gradient projection method. With the help of this reformulation we are able to prove strong convergence of the iterates. Moreover, we will give conditions under which the algorithm converges with linear speed [1]. These conditons are fulfilled, for example, for the case of ℓ 1 minimization problems. The viewpoint as a generalized gradient projection method will enable us to extend some parts of the theory to non-convex functionals R.

Abstract The proximity operator of a convex function is a natural extension of the notion of a projection operator onto a convex set. This tool, which plays a central role in the analysis and the numerical solution of convex optimization problems, has recently been introduced in the arena of inverse problems and, especially, in signal processing, where it has become increasingly important. In this paper, we review the basic properties of proximity operators which are relevant to signal processing and present optimization methods based on these operators. These proximal splitting methods are shown to capture and extend several well-known algorithms in a unifying framework. Applications of proximal methods in signal recovery and synthesis are discussed. Key words: Alternating-direction method of multipliers, backward-backward algorithm, convex optimization, denoising, Douglas-Rachford algorithm, forwardbackward algorithm, frame, Landweber method, iterative thresholding, parallel computing, Peaceman-Rachford algorithm, proximal algorithm, restoration and reconstruction,