Results 1  10
of
35
Decoding by Linear Programming
, 2004
"... This paper considers the classical error correcting problem which is frequently discussed in coding theory. We wish to recover an input vector f ∈ Rn from corrupted measurements y = Af + e. Here, A is an m by n (coding) matrix and e is an arbitrary and unknown vector of errors. Is it possible to rec ..."
Abstract

Cited by 662 (15 self)
 Add to MetaCart
This paper considers the classical error correcting problem which is frequently discussed in coding theory. We wish to recover an input vector f ∈ Rn from corrupted measurements y = Af + e. Here, A is an m by n (coding) matrix and e is an arbitrary and unknown vector of errors. Is it possible to recover f exactly from the data y? We prove that under suitable conditions on the coding matrix A, the input f is the unique solution to the ℓ1minimization problem (‖x‖ℓ1:= i xi) min g∈R n ‖y − Ag‖ℓ1 provided that the support of the vector of errors is not too large, ‖e‖ℓ0: = {i: ei ̸= 0}  ≤ ρ · m for some ρ> 0. In short, f can be recovered exactly by solving a simple convex optimization problem (which one can recast as a linear program). In addition, numerical experiments suggest that this recovery procedure works unreasonably well; f is recovered exactly even in situations where a significant fraction of the output is corrupted. This work is related to the problem of finding sparse solutions to vastly underdetermined systems of linear equations. There are also significant connections with the problem of recovering signals from highly incomplete measurements. In fact, the results introduced in this paper improve on our earlier work [5]. Finally, underlying the success of ℓ1 is a crucial property we call the uniform uncertainty principle that we shall describe in detail.
Using linear programming to decode binary linear codes
 IEEE TRANS. INFORM. THEORY
, 2005
"... A new method is given for performing approximate maximumlikelihood (ML) decoding of an arbitrary binary linear code based on observations received from any discrete memoryless symmetric channel. The decoding algorithm is based on a linear programming (LP) relaxation that is defined by a factor grap ..."
Abstract

Cited by 113 (11 self)
 Add to MetaCart
A new method is given for performing approximate maximumlikelihood (ML) decoding of an arbitrary binary linear code based on observations received from any discrete memoryless symmetric channel. The decoding algorithm is based on a linear programming (LP) relaxation that is defined by a factor graph or paritycheck representation of the code. The resulting “LP decoder” generalizes our previous work on turbolike codes. A precise combinatorial characterization of when the LP decoder succeeds is provided, based on pseudocodewords associated with the factor graph. Our definition of a pseudocodeword unifies other such notions known for iterative algorithms, including “stopping sets, ” “irreducible closed walks, ” “trellis cycles, ” “deviation sets, ” and “graph covers.” The fractional distance ��— ™ of a code is introduced, which is a lower bound on the classical distance. It is shown that the efficient LP decoder will correct up to ��— ™ P I errors and that there are codes with ��— ™ a @ I A. An efficient algorithm to compute the fractional distance is presented. Experimental evidence shows a similar performance on lowdensity paritycheck (LDPC) codes between LP decoding and the minsum and sumproduct algorithms. Methods for tightening the LP relaxation to improve performance are also provided.
Lowcomplexity approaches to SlepianWolf nearlossless distributed data compression
 IEEE TRANS. INFORM. THEORY
, 2006
"... This paper discusses the Slepian–Wolf problem of distributed nearlossless compression of correlated sources. We introduce practical new tools for communicating at all rates in the achievable region. The technique employs a simple “sourcesplitting” strategy that does not require common sources of ra ..."
Abstract

Cited by 23 (6 self)
 Add to MetaCart
This paper discusses the Slepian–Wolf problem of distributed nearlossless compression of correlated sources. We introduce practical new tools for communicating at all rates in the achievable region. The technique employs a simple “sourcesplitting” strategy that does not require common sources of randomness at the encoders and decoders. This approach allows for pipelined encoding and decoding so that the system operates with the complexity of a single user encoder and decoder. Moreover, when this splitting approach is used in conjunction with iterative decoding methods, it produces a significant simplification of the decoding process. We demonstrate this approach for synthetically generated data. Finally, we consider the Slepian–Wolf problem when linear codes are used as syndromeformers and consider a linear programming relaxation to maximumlikelihood (ML) sequence decoding. We note that the fractional vertices of the relaxed polytope compete with the optimal solution in a manner analogous to that observed when the “minsum ” iterative decoding algorithm is applied. This relaxation exhibits the MLcertificate property: if an integral solution is found, it is the ML solution. For symmetric binary joint distributions, we show that selecting easily constructable “expander”style lowdensity parity check codes (LDPCs) as syndromeformers admits a positive error exponent and therefore provably good performance.
Highly Robust Error Correction by Convex Programming
, 2006
"... This paper discusses a stylized communications problem where one wishes to transmit a realvalued signal x ∈ R n (a block of n pieces of information) to a remote receiver. We ask whether it is possible to transmit this information reliably when a fraction of the transmitted codeword is corrupted by ..."
Abstract

Cited by 20 (1 self)
 Add to MetaCart
This paper discusses a stylized communications problem where one wishes to transmit a realvalued signal x ∈ R n (a block of n pieces of information) to a remote receiver. We ask whether it is possible to transmit this information reliably when a fraction of the transmitted codeword is corrupted by arbitrary gross errors, and when in addition, all the entries of the codeword are contaminated by smaller errors (e.g. quantization errors). We show that if one encodes the information as Ax where A ∈ R m×n (m ≥ n) is a suitable coding matrix, there are two decoding schemes that allow the recovery of the block of n pieces of information x with nearly the same accuracy as if no gross errors occur upon transmission (or equivalently as if one has an oracle supplying perfect information about the sites and amplitudes of the gross errors). Moreover, both decoding strategies are very concrete and only involve solving simple convex optimization programs, either a linear program or a secondorder cone program. We complement our study with numerical simulations showing that the encoder/decoder pair performs remarkably well.
LP Decoding Achieves Capacity
 In SODA
, 2004
"... We give a linear programming (LP) decoder that achieves the capacity (optimal rate) of a wide range of probabilistic binary communication channels. This is the first such result for LP decoding. More generally, as far as the authors are aware this is the first known polynomialtime capacityachiev ..."
Abstract

Cited by 20 (3 self)
 Add to MetaCart
We give a linear programming (LP) decoder that achieves the capacity (optimal rate) of a wide range of probabilistic binary communication channels. This is the first such result for LP decoding. More generally, as far as the authors are aware this is the first known polynomialtime capacityachieving decoder with the maximumlikelihood (ML) certificate propertywhere output codewords come with a proof of optimality.
Dense error correction via ℓ1 minimization
, 2009
"... This paper studies the problem of recovering a nonnegative sparse signal x ∈ Rn from highly corrupted linear measurements y = Ax + e ∈ Rm, where e is an unknown error vector whose nonzero entries may be unbounded. Motivated by an observation from face recognition in computer vision, this paper prov ..."
Abstract

Cited by 17 (5 self)
 Add to MetaCart
This paper studies the problem of recovering a nonnegative sparse signal x ∈ Rn from highly corrupted linear measurements y = Ax + e ∈ Rm, where e is an unknown error vector whose nonzero entries may be unbounded. Motivated by an observation from face recognition in computer vision, this paper proves that for highly correlated (and possibly overcomplete) dictionaries A, any nonnegative, sufficiently sparse signal x can be recovered by solving an ℓ1minimization problem: min ‖x‖1 + ‖e‖1 subject to y = Ax + e. More precisely, if the fraction ρ of errors is bounded away from one and the support of x grows sublinearly in the dimension m of the observation, then as m goes to infinity, the above ℓ1minimization succeeds for all signals x and almost all signandsupport patterns of e. This result suggests that accurate recovery of sparse signals is possible and computationally feasible even with nearly 100 % of the observations corrupted. The proof relies on a careful characterization of the faces of a convex polytope spanned together by the standard crosspolytope and a set of iid Gaussian vectors with nonzero mean and small variance, which we call the “crossandbouquet ” model. Simulations and experimental results corroborate the findings, and suggest extensions to the result.
Calderbank R., Efficient and Robust Compressive Sensing using HighQuality Expander Graphs. Submitted to the IEEE transaction on Information Theory
, 2008
"... Abstract—Expander graphs have been recently proposed to construct efficient compressed sensing algorithms. In particular, it has been shown that any ndimensional vector that is ksparse (with k ≪ n) can be fully recovered using O(k log n k) measurements and only O(k log n) simple recovery iteration ..."
Abstract

Cited by 12 (1 self)
 Add to MetaCart
Abstract—Expander graphs have been recently proposed to construct efficient compressed sensing algorithms. In particular, it has been shown that any ndimensional vector that is ksparse (with k ≪ n) can be fully recovered using O(k log n k) measurements and only O(k log n) simple recovery iterations. In this paper we improve upon this result by considering expander graphs with expansion coefficient beyond 3 and show that, with 4 the same number of measurements, only O(k) recovery iterations are required, which is a significant improvement when n is large. In fact, full recovery can be accomplished by at most 2k very simple iterations. The number of iterations can be made arbitrarily close to k, and the recovery algorithm can be implemented very efficiently using a simple binary search tree. We also show that by tolerating a small penalty on the number of measurements, and not on the number of recovery iterations, one can use the efficient construction of a family of expander graphs to come up with explicit measurement matrices for this method. We compare our result with other recently developed expandergraphbased methods and argue that it compares favorably both in terms of the number of required measurements and in terms of the recovery time complexity. Finally we will show how our analysis extends to give a robust algorithm that finds the position and sign of the k significant elements of an almost ksparse signal and then, using very simple optimization techniques, finds in sublinear time a ksparse signal which approximates the original signal with very high precision. I.
Guessing Facets: Polytope Structure and Improved
 LP Decoder, Proceedings of the IEEE ISIT, Seattle 2006, arxiv:cs/0608029
, 2006
"... Abstract—We investigate the structure of the polytope underlying the linear programming (LP) decoder introduced by Feldman, Karger, and Wainwright. We first show that for expander codes, every fractional pseudocodeword always has at least a constant fraction of nonintegral bits. We then prove that f ..."
Abstract

Cited by 12 (0 self)
 Add to MetaCart
Abstract—We investigate the structure of the polytope underlying the linear programming (LP) decoder introduced by Feldman, Karger, and Wainwright. We first show that for expander codes, every fractional pseudocodeword always has at least a constant fraction of nonintegral bits. We then prove that for expander codes, the active set of any fractional pseudocodeword is smaller by a constant fraction than that of any codeword. We further exploit these geometrical properties to devise an improved decoding algorithm with the same order of complexity as LP decoding that provably performs better. The method is very simple: it first applies ordinary LP decoding, and when it fails, it proceeds by guessing facets of the polytope, and then resolving the linear program on these facets. While the LP decoder succeeds only if the ML codeword has the highest likelihood over all pseudocodewords, we prove that the proposed algorithm, when applied to suitable expander codes, succeeds unless there exists a certain number of pseudocodewords, all adjacent to the ML codeword on the LP decoding polytope, and with higher likelihood than the ML codeword. We then describe an extended algorithm, still with polynomial complexity, that succeeds as long as there are at most polynomially many pseudocodewords above the ML codeword. Index Terms—Belief propagation, bit guessing, decimation, errorcorrecting codes, iterative decoding, linear programming (LP), lowdensity paritycheck (LDPC) codes, linear programming (LP) decoding, pseudocodewords. I.
TreewidthBased Conditions for Exactness Of The SheraliAdams and . . .
, 2004
"... The SheraliAdams (SA) and Lasserre (LS) approaches are "liftandproject" methods that generate nested sequences of linear and/or semidefinite relaxations of an arbitrary 01 polytope . Although both procedures are known to terminate with an exact description of P after n steps, there are va ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
The SheraliAdams (SA) and Lasserre (LS) approaches are "liftandproject" methods that generate nested sequences of linear and/or semidefinite relaxations of an arbitrary 01 polytope . Although both procedures are known to terminate with an exact description of P after n steps, there are various open questions associated with characterizing, for particular problem classes, whether exactness is obtained at some step s n. This paper provides su# cient conditions for exactness of these relaxations based on the hypergraphtheoretic notion of treewidth. More specifically, we relate the combinatorial structure of a given polynomial system to an underlying hypergraph. We prove that the complexity of assessing the global validity of moment sequences, and hence the tightness of the SA and LS relaxations, is determined by the treewidth of this hypergraph. We provide some examples to illustrate this characterization.