Results 1 - 10
of
46
Information-Theoretically Optimal Compressed Sensing via Spatial Coupling and Approximate Message Passing
, 2011
"... We study the compressed sensing reconstruction problem for a broad class of random, banddiagonal sensing matrices. This construction is inspired by the idea of spatial coupling in coding theory. As demonstrated heuristically and numerically by Krzakala et al. [KMS+ 11], message passing algorithms ca ..."
Abstract
-
Cited by 51 (5 self)
- Add to MetaCart
(Show Context)
We study the compressed sensing reconstruction problem for a broad class of random, banddiagonal sensing matrices. This construction is inspired by the idea of spatial coupling in coding theory. As demonstrated heuristically and numerically by Krzakala et al. [KMS+ 11], message passing algorithms can effectively solve the reconstruction problem for spatially coupled measurements with undersampling rates close to the fraction of non-zero coordinates. We use an approximate message passing (AMP) algorithm and analyze it through the state evolution method. We give a rigorous proof that this approach is successful as soon as the undersampling rate δ exceeds the (upper) Rényi information dimension of the signal, d(pX). More precisely, for a sequence of signals of diverging dimension n whose empirical distribution converges to pX, reconstruction is with high probability successful from d(pX) n + o(n) measurements taken according to a band diagonal matrix. For sparse signals, i.e. sequences of dimension n and k(n) non-zero entries, this implies reconstruction from k(n)+o(n) measurements. For ‘discrete ’ signals, i.e. signals whose coordinates take a fixed finite set of values, this implies reconstruction from o(n) measurements. The result
A simple proof of threshold saturation for coupled scalar recursions
- in Proc. Intl. Symp. on Turbo Codes and Iter. Inform. Proc. (ISTC), 2012
"... ar ..."
(Show Context)
Optimal phase transitions in compressed sensing
- IEEE TRANS. INF. THEORY
, 2012
"... Compressed sensing deals with efficient recovery of analog signals from linear encodings. This paper presents a statistical study of compressed sensing by modeling the input signal as an i.i.d. process with known distribution. Three classes of encoders are considered, namely optimal nonlinear, opti ..."
Abstract
-
Cited by 26 (3 self)
- Add to MetaCart
Compressed sensing deals with efficient recovery of analog signals from linear encodings. This paper presents a statistical study of compressed sensing by modeling the input signal as an i.i.d. process with known distribution. Three classes of encoders are considered, namely optimal nonlinear, optimal linear, and random linear encoders. Focusing on optimal decoders, we investigate the fundamental tradeoff between measurement rate and reconstruction fidelity gauged by error probability and noise sensitivity in the absence and presence of measurement noise, respectively. The optimal phase-transition threshold is determined as a functional of the input distribution and compared to suboptimal thresholds achieved by popular reconstruction algorithms. In particular, we show that Gaussian sensing matrices incur no penalty on the phase-transition threshold with respect to optimal nonlinear encoding. Our results also provide a rigorous justification of previous results based on replica heuristics in the weak-noise regime.
State Evolution for General Approximate Message Passing Algorithms, with Applications to Spatial Coupling
, 2012
"... We consider a class of approximated message passing (AMP) algorithms and characterize their high-dimensional behavior in terms of a suitable state evolution recursion. Our proof applies to Gaussian matrices with independent but not necessarily identically distributed entries. It covers – in particul ..."
Abstract
-
Cited by 23 (7 self)
- Add to MetaCart
(Show Context)
We consider a class of approximated message passing (AMP) algorithms and characterize their high-dimensional behavior in terms of a suitable state evolution recursion. Our proof applies to Gaussian matrices with independent but not necessarily identically distributed entries. It covers – in particular – the analysis of generalized AMP, introduced by Rangan, and of AMP reconstruction in compressed sensing with spatially coupled sensing matrices. The proof technique builds on the one of [BM11], while simplifying and generalizing several steps. 1
Spatially-coupled random access on graphs
- in Proc. of IEEE ISIT 2012
, 2012
"... ar ..."
(Show Context)
Subsampling at information theoretically optimal rates
- IEEE Intl. Symp. on Inform. Theory
, 2012
"... Abstract—We study the problem of sampling a random signal with sparse support in frequency domain. Shannon famously considered a scheme that instantaneously samples the signal at equispaced times. He proved that the signal can be reconstructed as long as the sampling rate exceeds twice the bandwidth ..."
Abstract
-
Cited by 10 (2 self)
- Add to MetaCart
Abstract—We study the problem of sampling a random signal with sparse support in frequency domain. Shannon famously considered a scheme that instantaneously samples the signal at equispaced times. He proved that the signal can be reconstructed as long as the sampling rate exceeds twice the bandwidth (Nyquist rate). Candès, Romberg, Tao introduced a scheme that acquires instantaneous samples of the signal at random times. They proved that the signal can be uniquely and efficiently reconstructed, provided the sampling rate exceeds the frequency support of the signal, times logarithmic factors. In this paper we consider a probabilistic model for the signal, and a sampling scheme inspired by the idea of spatial coupling in coding theory. Namely, we propose to acquire non-instantaneous samples at random times. Mathematically, this is implemented by acquiring a small random subset of Gabor coefficients. We show empirically that this scheme achieves correct reconstruction as soon as the sampling rate exceeds the frequency support of the signal, thus reaching the information theoretic limit. I.
Universal codes for the gaussian mac via spatial coupling
- in Proc. 49th Ann. Allerton Conf. Comm. Control Comput
"... Abstract—We consider transmission of two independent and separately encoded sources over a two-user binary-input Gaus-sian multiple-access channel. The channel gains are assumed to be unknown at the transmitter and the goal is to design an encoder-decoder pair that achieves reliable communication fo ..."
Abstract
-
Cited by 8 (2 self)
- Add to MetaCart
(Show Context)
Abstract—We consider transmission of two independent and separately encoded sources over a two-user binary-input Gaus-sian multiple-access channel. The channel gains are assumed to be unknown at the transmitter and the goal is to design an encoder-decoder pair that achieves reliable communication for all channel gains where this is theoretically possible. We call such a system universal with respect to the channel gains. Kudekar et al. recently showed that terminated low-density parity-check convolutional codes (a.k.a. spatially-coupled low-density parity-check ensembles) have belief-propagation thresh-olds that approach their maximum a-posteriori thresholds. This was proven for binary erasure channels and shown empirically for binary memoryless symmetric channels. It was conjectured that the principle of spatial coupling is very general and the phenomenon of threshold saturation applies to a very broad class of graphical models. In this work, we derive an area theorem for the joint decoder and empirically show that threshold saturation occurs for this problem. As a result, we demonstrate near-universal performance for this problem using the proposed spatially-coupled coding system. Index Terms—Gaussian MAC, LDPC codes, spatial coupling, EXIT functions, density evolution, joint decoding, protograph, area theorem. I.
