Results 1  10
of
24
MAP estimation via agreement on trees: Messagepassing and linear programming
, 2002
"... We develop and analyze methods for computing provably optimal maximum a posteriori (MAP) configurations for a subclass of Markov random fields defined on graphs with cycles. By decomposing the original distribution into a convex combination of treestructured distributions, we obtain an upper bound ..."
Abstract

Cited by 132 (8 self)
 Add to MetaCart
We develop and analyze methods for computing provably optimal maximum a posteriori (MAP) configurations for a subclass of Markov random fields defined on graphs with cycles. By decomposing the original distribution into a convex combination of treestructured distributions, we obtain an upper bound on the optimal value of the original problem (i.e., the log probability of the MAP assignment) in terms of the combined optimal values of the tree problems. We prove that this upper bound is tight if and only if all the tree distributions share an optimal configuration in common. An important implication is that any such shared configuration must also be a MAP configuration for the original distribution. Next we develop two approaches to attempting to obtain tight upper bounds: (a) a treerelaxed linear program (LP), which is derived from the Lagrangian dual of the upper bounds; and (b) a treereweighted maxproduct messagepassing algorithm that is related to but distinct from the maxproduct algorithm. In this way, we establish a connection between a certain LP relaxation of the modefinding problem, and a reweighted form of the maxproduct (minsum) messagepassing algorithm.
Using linear programming to decode binary linear codes
 IEEE TRANS. INFORM. THEORY
, 2005
"... A new method is given for performing approximate maximumlikelihood (ML) decoding of an arbitrary binary linear code based on observations received from any discrete memoryless symmetric channel. The decoding algorithm is based on a linear programming (LP) relaxation that is defined by a factor grap ..."
Abstract

Cited by 113 (11 self)
 Add to MetaCart
A new method is given for performing approximate maximumlikelihood (ML) decoding of an arbitrary binary linear code based on observations received from any discrete memoryless symmetric channel. The decoding algorithm is based on a linear programming (LP) relaxation that is defined by a factor graph or paritycheck representation of the code. The resulting “LP decoder” generalizes our previous work on turbolike codes. A precise combinatorial characterization of when the LP decoder succeeds is provided, based on pseudocodewords associated with the factor graph. Our definition of a pseudocodeword unifies other such notions known for iterative algorithms, including “stopping sets, ” “irreducible closed walks, ” “trellis cycles, ” “deviation sets, ” and “graph covers.” The fractional distance ��— ™ of a code is introduced, which is a lower bound on the classical distance. It is shown that the efficient LP decoder will correct up to ��— ™ P I errors and that there are codes with ��— ™ a @ I A. An efficient algorithm to compute the fractional distance is presented. Experimental evidence shows a similar performance on lowdensity paritycheck (LDPC) codes between LP decoding and the minsum and sumproduct algorithms. Methods for tightening the LP relaxation to improve performance are also provided.
TreeBased Reparameterization Framework for Analysis of Belief Propagation and Related Algorithms
, 2001
"... We present a treebased reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sumproduct algorithm [39, 36], as well as a rich set of variations and ext ..."
Abstract

Cited by 102 (22 self)
 Add to MetaCart
We present a treebased reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sumproduct algorithm [39, 36], as well as a rich set of variations and extensions of belief propagation. Algorithms in this class can be formulated as a sequence of reparameterization updates, each of which entails refactorizing a portion of the distribution corresponding to an acyclic subgraph (i.e., a tree). The ultimate goal is to obtain an alternative but equivalent factorization using functions that represent (exact or approximate) marginal distributions on cliques of the graph. Our framework highlights an important property of BP and the entire class of reparameterization algorithms: the distribution on the full graph is not changed. The perspective of treebased updates gives rise to a simple and intuitive characterization of the fixed points in terms of tree consistency. We develop interpretations of these results in terms of information geometry. The invariance of the distribution, in conjunction with the fixed point characterization, enables us to derive an exact relation between the exact marginals on an arbitrary graph with cycles, and the approximations provided by belief propagation, and more broadly, any algorithm that minimizes the Bethe free energy. We also develop bounds on this approximation error, which illuminate the conditions that govern their accuracy. Finally, we show how the reparameterization perspective extends naturally to more structured approximations (e.g., Kikuchi and variants [52, 37]) that operate over higher order cliques.
Bayesian Compressed Sensing via Belief Propagation
, 2010
"... Compressive sensing (CS) is an emerging field based on the revelation that a small collection of linear projections of a sparse signal contains enough information for stable, subNyquist signal acquisition. When a statistical characterization of the signal is available, Bayesian inference can comple ..."
Abstract

Cited by 51 (12 self)
 Add to MetaCart
Compressive sensing (CS) is an emerging field based on the revelation that a small collection of linear projections of a sparse signal contains enough information for stable, subNyquist signal acquisition. When a statistical characterization of the signal is available, Bayesian inference can complement conventional CS methods based on linear programming or greedy algorithms. We perform asymptotically optimal Bayesian inference using belief propagation (BP) decoding, which represents the CS encoding matrix as a graphical model. Fast computation is obtained by reducing the size of the graphical model with sparse encoding matrices. To decode a length signal containing large coefficients, our CSBP decoding algorithm uses ( log ()) measurements and ( log 2 ()) computation. Finally, although we focus on a twostate mixture Gaussian model, CSBP is easily adapted to other signal models.
Maximum weight matching via maxproduct belief propagation
 in International Symposium of Information Theory
, 2005
"... Abstract — The maxproduct “belief propagation ” algorithm is an iterative, local, message passing algorithm for finding the maximum a posteriori (MAP) assignment of a discrete probability distribution specified by a graphical model. Despite the spectacular success of the algorithm in many applicati ..."
Abstract

Cited by 46 (7 self)
 Add to MetaCart
Abstract — The maxproduct “belief propagation ” algorithm is an iterative, local, message passing algorithm for finding the maximum a posteriori (MAP) assignment of a discrete probability distribution specified by a graphical model. Despite the spectacular success of the algorithm in many application areas such as iterative decoding and computer vision which involve graphs with many cycles, theoretical convergence results are only known for graphs which are treelike or have a single cycle. In this paper, we consider a weighted complete bipartite graph and define a probability distribution on it whose MAP assignment corresponds to the maximum weight matching (MWM) in that graph. We analyze the fixed points of the maxproduct algorithm when run on this graph and prove the surprising result that even though the underlying graph has many short cycles, the maxproduct assignment converges to the correct MAP assignment. We also provide a bound on the number of iterations required by the algorithm. I.
Compressed Sensing Reconstruction via Belief Propagation
, 2006
"... Compressed sensing is an emerging field that enables to reconstruct sparse or compressible signals from a small number of linear projections. We describe a specific measurement scheme using an LDPClike measurement matrix, which is a realvalued analogue to LDPC techniques over a finite alphabet. We ..."
Abstract

Cited by 39 (8 self)
 Add to MetaCart
Compressed sensing is an emerging field that enables to reconstruct sparse or compressible signals from a small number of linear projections. We describe a specific measurement scheme using an LDPClike measurement matrix, which is a realvalued analogue to LDPC techniques over a finite alphabet. We then describe the reconstruction details for mixture Gaussian signals. The technique can be extended to additional compressible signal models. 1
Sudocodes  Fast Measurement and Reconstruction of Sparse Signals
 Proc. IEEE Int. Symposium on Information Theory (ISIT
, 2006
"... Abstract — Sudocodes are a new scheme for lossless compressive sampling and reconstruction of sparse signals. Consider a sparse signal x ∈ R N containing only K ≪ N nonzero values. Sudoencoding computes the codeword y ∈ R M via the linear matrixvector multiplication y = Φx, with K < M ≪ N. We pro ..."
Abstract

Cited by 31 (2 self)
 Add to MetaCart
Abstract — Sudocodes are a new scheme for lossless compressive sampling and reconstruction of sparse signals. Consider a sparse signal x ∈ R N containing only K ≪ N nonzero values. Sudoencoding computes the codeword y ∈ R M via the linear matrixvector multiplication y = Φx, with K < M ≪ N. We propose a nonadaptive construction of a sparse Φ comprising only the values 0 and 1; hence the computation of y involves only sums of subsets of the elements of x. An accompanying sudodecoding strategy efficiently recovers x given y. Sudocodes require only M = O(K log(N)) measurements for exact reconstruction with worstcase computational complexity O(K log(K) log(N)). Sudocodes can be used as erasure codes for realvalued data and have potential applications in peertopeer networks and distributed data storage systems. They are also easily extended to signals that are sparse in arbitrary bases. I.
Analysis of lowdensity paritycheck codes for the GilbertElliott channel
 IEEE TRANS. INF. THEORY
, 2005
"... Density evolution analysis of lowdensity paritycheck (LDPC) codes in memoryless channels is extended to the Gilbert–Elliott (GE) channel, which is a special case of a large class of channels with hidden Markov memory. In a procedure referred to as estimation decoding, the sum–product algorithm (S ..."
Abstract

Cited by 28 (6 self)
 Add to MetaCart
Density evolution analysis of lowdensity paritycheck (LDPC) codes in memoryless channels is extended to the Gilbert–Elliott (GE) channel, which is a special case of a large class of channels with hidden Markov memory. In a procedure referred to as estimation decoding, the sum–product algorithm (SPA) is used to perform LDPC decoding jointly with channelstate detection. Density evolution results show (and simulation results confirm) that such decoders provide a significantly enlarged region of successful decoding within the GE parameter space, compared with decoders that do not exploit the channel memory. By considering a variety of ways in which a GE channel may be degraded, it is shown how knowledge of the decoding behavior at a single point of the GE parameter space may be extended to a larger region within the space, thereby mitigating the large complexity needed in using density evolution to explore the parameter space pointbypoint. Using the GE channel as a straightforward example, we conclude that analysis of estimation decoding for LDPC codes is feasible in channels with memory, and that such analysis shows large potential gains.
BeliefPropagation for Weighted bMatchings on Arbitrary Graphs and its Relation to Linear Programs with Integer Solutions
 in arXiv, http://www.arxiv.org/abs/0709.1190v1
, 2007
"... We consider the general problem of finding the minimum weight bmatching on arbitrary graphs. We prove that, whenever the linear programming (LP) relaxation of the problem has no fractional solutions, then the belief propagation (BP) algorithm converges to the correct solution. This result is notabl ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
We consider the general problem of finding the minimum weight bmatching on arbitrary graphs. We prove that, whenever the linear programming (LP) relaxation of the problem has no fractional solutions, then the belief propagation (BP) algorithm converges to the correct solution. This result is notable in several regards: (1) It is one of a very small number of proofs showing correctness of BP without any constraint on the graph structure. (2) Instead of showing that BP leads to a PTAS, we give a finite bound for the number of iterations after which BP has converged to the exact solution. (3) Variants of the proof work for both synchronous and asynchronous BP; to the best of our knowledge, it is the first proof of convergence and correctness of an asynchronous BP algorithm for a combinatorial optimization problem. (4) It works for both ordinary bmatchings and the more difficult case of perfect bmatchings. (5) Together with the recent work of Sanghavi, Malioutov and Wilskly [41] they are the first complete proofs showing that tightness of LP implies correctness of BP. 1