Abstract. We introduce a new iterative algorithm to find sparse solutions of underdetermined linear systems. The algorithm, a simple combination of the Iterative Hard Thresholding algorithm and of the Compressive Sampling Matching Pursuit or Subspace Pursuit algorithms, is called Hard Thresholding Pursuit. We study its general convergence, and notice in particular that only a finite number of iterations are required. We then show that, under a certain condition on the restricted isometry constant of the matrix of the system, the Hard Thresholding Pursuit algorithm indeed finds all s-sparse solutions. This condition, which reads δ3s < 1 / √ 3, is heuristically better than the sufficient conditions currently available for other Compressive Sensing algorithms. It applies to fast versions of the algorithm, too, including the Iterative Hard Thresholding algorithm. Stability with respect to sparsity defect and robustness with respect to measurement error are also guaranteed under the condition δ3s < 1 / √ 3. We conclude with some numerical experiments to demonstrate the good empirical performance and the low complexity of the Hard Thresholding Pursuit algorithm.

We propose and analyze acceleration schemes for hard thresholding methods with applications to sparse approximation in linear inverse systems. Our acceleration schemes fuse combinatorial, sparse projection algorithms with convex optimization algebra to provide computationally efficient and robust sparse recovery methods. We compare and contrast the (dis)advantages of the proposed schemes with the state-of-the-art, not only within hard thresholding methods, but also within convex sparse recovery algorithms. 1

Recovering a sparse signal from an undersampled set of random linear measurements is the main problem of interest in compressed sensing. In this paper, we consider the case where both the signal and the measurements are complex-valued. We study the popular reconstruction method of ℓ1-regularized least squares or LASSO. While several studies have shown that the LASSO algorithm offers desirable solutions under certain conditions, the precise asymptotic performance of this algorithm in the complex setting is not yet known. In this paper, we extend the approximate message passing (AMP) algorithm to the complex-valued signals and measurements to obtain the complex approximate message passing algorithm (CAMP). We then generalize the state evolution framework recently introduced for the analysis of AMP, to the complex setting. Using the state evolution, we derive accurate formulas for the phase transition and noise sensitivity of both LASSO and CAMP. Our results are theoretically proved for the case of i.i.d. Gaussian sensing matrices. But we confirm through simulations that our results hold for larger class of random matrices. 1

We provide two compressive sensing (CS) recovery algorithms based on iterative hard-thresholding. The algorithms, collectively dubbed as algebraic pursuits (ALPS), exploit the restricted isometry properties of the CS measurement matrix within the algebra of Nesterov’s optimal gradient methods. We theoretically characterize the approximation guarantees of ALPS for signals that are sparse on ortho-bases as well as on tight-frames. Simulation results demonstrate a great potential for ALPS in terms of phase-transition, noise robustness, and CS reconstruction. 1.

This document presents details concerning analytical derivations and numerical experiments that support the claims made in the main text ‘Message Passing Algorithms for Compressed Sensing’, submitted for publication in the Proceedings of the National Academy of Sciences, USA. Hereafter ’main text’. One can find here: • Derivations of explicit Formulas for the MSE Map, and the optimal thresholds; see Section 3 below. • Proof of Theorem 1; see Section 3 below. • Proof of concavity of the MSE Map, see again Section 3 below. • Explanation of the connection between Minimax Thresholding, Minimax Risk, and rigorous proof of formula [19] in the main text; see Section 4 below, Theorem 4.2. • Formulas for the rate exponent b of Theorem 2 in the main text, expressed in terms of the minimax threshold risk; see Section 5 below.

We provide an algorithmic framework for structured sparse recovery which unifies combinatorial optimization with the non-smooth convex optimization framework by Nesterov [1, 2]. Our algorithm, dubbed Nesterov iterative hard-thresholding (NIHT), is similar to the algebraic pursuits (ALPS) in [3] in spirit: we use the gradient information in the convex data error objective to navigate over the nonconvex set of structured sparse signals. While ALPS feature a priori approximation guarantees, we were only able to provide an online approximation guarantee for NIHT (e.g., the guarantees require the algorithm execution). Experiments show however that NIHT can empirically outperform ALPS and other state-ofthe-art convex optimization-based algorithms in sparse recovery. 1

We consider the problem of target detection from a set of Compressive Sensing (CS) radar measurements corrupted by additive white Gaussian noise. We propose two novel architectures and compare their performance by means of Receiver Operating Characteristic (ROC) curves. Using asymptotic arguments and the Complex Approximate Message Passing (CAMP) algorithm, we characterize the statistics of the ℓ1-norm reconstruction error and derive closed form expressions for both the detection and false alarm probabilities of both schemes. Of the two architectures, we demonstrate that the best one consists of a reconstruction stage based on CAMP followed by a detector. This architecture, which outperforms the ℓ1-based detector in the ideal case of known background noise, can also be made fully adaptive by combining it with a conventional Constant False Alarm Rate (CFAR) processor. Using the state evolution framework of CAMP, we also derive Signal to Noise Ratio (SNR) maps that, together with the ROC curves, can be used to design a CS-based CFAR radar detector. Our theoretical findings are confirmed by means of both Monte Carlo simulations and experimental results.

The nascent field of compressed sensing is founded on the fact that high-dimensional signals with “simple structure ” can be recovered accurately from just a small number of randomized samples. Several specific kinds of structures have been explored in the literature, from sparsity and group sparsity to low-rankedness. However, two fundamental questions have been left unanswered, namely: What are the general abstract meanings of “structure ” and “simplicity”? And do there exist universal algorithms for recovering such simple structured objects from fewer samples than their ambient dimension? In this paper, we address these two questions. Using algorithmic information theory tools such as the Kolmogorov complexity, we provide a unified definition of structure and simplicity. Leveraging this new definition, we develop and analyze an abstract algorithm for signal recovery motivated by Occam’s Razor. Minimum complexity pursuit (MCP) requires just O(κ log n) randomized samples to recover a signal of complexity κ and ambient dimension n. We also discuss the performance of MCP in the presence of measurement noise and with approximately simple signals. I.