Results 1  10
of
46
InformationTheoretically Optimal Compressed Sensing via Spatial Coupling and Approximate Message Passing
, 2011
"... We study the compressed sensing reconstruction problem for a broad class of random, banddiagonal sensing matrices. This construction is inspired by the idea of spatial coupling in coding theory. As demonstrated heuristically and numerically by Krzakala et al. [KMS+ 11], message passing algorithms ca ..."
Abstract

Cited by 51 (5 self)
 Add to MetaCart
(Show Context)
We study the compressed sensing reconstruction problem for a broad class of random, banddiagonal sensing matrices. This construction is inspired by the idea of spatial coupling in coding theory. As demonstrated heuristically and numerically by Krzakala et al. [KMS+ 11], message passing algorithms can effectively solve the reconstruction problem for spatially coupled measurements with undersampling rates close to the fraction of nonzero coordinates. We use an approximate message passing (AMP) algorithm and analyze it through the state evolution method. We give a rigorous proof that this approach is successful as soon as the undersampling rate δ exceeds the (upper) Rényi information dimension of the signal, d(pX). More precisely, for a sequence of signals of diverging dimension n whose empirical distribution converges to pX, reconstruction is with high probability successful from d(pX) n + o(n) measurements taken according to a band diagonal matrix. For sparse signals, i.e. sequences of dimension n and k(n) nonzero entries, this implies reconstruction from k(n)+o(n) measurements. For ‘discrete ’ signals, i.e. signals whose coordinates take a fixed finite set of values, this implies reconstruction from o(n) measurements. The result
A simple proof of threshold saturation for coupled scalar recursions
 in Proc. Intl. Symp. on Turbo Codes and Iter. Inform. Proc. (ISTC), 2012
"... ar ..."
(Show Context)
Optimal phase transitions in compressed sensing
 IEEE TRANS. INF. THEORY
, 2012
"... Compressed sensing deals with efficient recovery of analog signals from linear encodings. This paper presents a statistical study of compressed sensing by modeling the input signal as an i.i.d. process with known distribution. Three classes of encoders are considered, namely optimal nonlinear, opti ..."
Abstract

Cited by 26 (3 self)
 Add to MetaCart
Compressed sensing deals with efficient recovery of analog signals from linear encodings. This paper presents a statistical study of compressed sensing by modeling the input signal as an i.i.d. process with known distribution. Three classes of encoders are considered, namely optimal nonlinear, optimal linear, and random linear encoders. Focusing on optimal decoders, we investigate the fundamental tradeoff between measurement rate and reconstruction fidelity gauged by error probability and noise sensitivity in the absence and presence of measurement noise, respectively. The optimal phasetransition threshold is determined as a functional of the input distribution and compared to suboptimal thresholds achieved by popular reconstruction algorithms. In particular, we show that Gaussian sensing matrices incur no penalty on the phasetransition threshold with respect to optimal nonlinear encoding. Our results also provide a rigorous justification of previous results based on replica heuristics in the weaknoise regime.
State Evolution for General Approximate Message Passing Algorithms, with Applications to Spatial Coupling
, 2012
"... We consider a class of approximated message passing (AMP) algorithms and characterize their highdimensional behavior in terms of a suitable state evolution recursion. Our proof applies to Gaussian matrices with independent but not necessarily identically distributed entries. It covers – in particul ..."
Abstract

Cited by 23 (7 self)
 Add to MetaCart
(Show Context)
We consider a class of approximated message passing (AMP) algorithms and characterize their highdimensional behavior in terms of a suitable state evolution recursion. Our proof applies to Gaussian matrices with independent but not necessarily identically distributed entries. It covers – in particular – the analysis of generalized AMP, introduced by Rangan, and of AMP reconstruction in compressed sensing with spatially coupled sensing matrices. The proof technique builds on the one of [BM11], while simplifying and generalizing several steps. 1
Spatiallycoupled random access on graphs
 in Proc. of IEEE ISIT 2012
, 2012
"... ar ..."
(Show Context)
Subsampling at information theoretically optimal rates
 IEEE Intl. Symp. on Inform. Theory
, 2012
"... Abstract—We study the problem of sampling a random signal with sparse support in frequency domain. Shannon famously considered a scheme that instantaneously samples the signal at equispaced times. He proved that the signal can be reconstructed as long as the sampling rate exceeds twice the bandwidth ..."
Abstract

Cited by 10 (2 self)
 Add to MetaCart
Abstract—We study the problem of sampling a random signal with sparse support in frequency domain. Shannon famously considered a scheme that instantaneously samples the signal at equispaced times. He proved that the signal can be reconstructed as long as the sampling rate exceeds twice the bandwidth (Nyquist rate). Candès, Romberg, Tao introduced a scheme that acquires instantaneous samples of the signal at random times. They proved that the signal can be uniquely and efficiently reconstructed, provided the sampling rate exceeds the frequency support of the signal, times logarithmic factors. In this paper we consider a probabilistic model for the signal, and a sampling scheme inspired by the idea of spatial coupling in coding theory. Namely, we propose to acquire noninstantaneous samples at random times. Mathematically, this is implemented by acquiring a small random subset of Gabor coefficients. We show empirically that this scheme achieves correct reconstruction as soon as the sampling rate exceeds the frequency support of the signal, thus reaching the information theoretic limit. I.
Universal codes for the gaussian mac via spatial coupling
 in Proc. 49th Ann. Allerton Conf. Comm. Control Comput
"... Abstract—We consider transmission of two independent and separately encoded sources over a twouser binaryinput Gaussian multipleaccess channel. The channel gains are assumed to be unknown at the transmitter and the goal is to design an encoderdecoder pair that achieves reliable communication fo ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
(Show Context)
Abstract—We consider transmission of two independent and separately encoded sources over a twouser binaryinput Gaussian multipleaccess channel. The channel gains are assumed to be unknown at the transmitter and the goal is to design an encoderdecoder pair that achieves reliable communication for all channel gains where this is theoretically possible. We call such a system universal with respect to the channel gains. Kudekar et al. recently showed that terminated lowdensity paritycheck convolutional codes (a.k.a. spatiallycoupled lowdensity paritycheck ensembles) have beliefpropagation thresholds that approach their maximum aposteriori thresholds. This was proven for binary erasure channels and shown empirically for binary memoryless symmetric channels. It was conjectured that the principle of spatial coupling is very general and the phenomenon of threshold saturation applies to a very broad class of graphical models. In this work, we derive an area theorem for the joint decoder and empirically show that threshold saturation occurs for this problem. As a result, we demonstrate nearuniversal performance for this problem using the proposed spatiallycoupled coding system. Index Terms—Gaussian MAC, LDPC codes, spatial coupling, EXIT functions, density evolution, joint decoding, protograph, area theorem. I.