Results 1  10
of
34
TreeBased Reparameterization Framework for Analysis of Belief Propagation and Related Algorithms
, 2001
"... We present a treebased reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sumproduct algorithm [39, 36], as well as a rich set of variations and ext ..."
Abstract

Cited by 102 (21 self)
 Add to MetaCart
We present a treebased reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sumproduct algorithm [39, 36], as well as a rich set of variations and extensions of belief propagation. Algorithms in this class can be formulated as a sequence of reparameterization updates, each of which entails refactorizing a portion of the distribution corresponding to an acyclic subgraph (i.e., a tree). The ultimate goal is to obtain an alternative but equivalent factorization using functions that represent (exact or approximate) marginal distributions on cliques of the graph. Our framework highlights an important property of BP and the entire class of reparameterization algorithms: the distribution on the full graph is not changed. The perspective of treebased updates gives rise to a simple and intuitive characterization of the fixed points in terms of tree consistency. We develop interpretations of these results in terms of information geometry. The invariance of the distribution, in conjunction with the fixed point characterization, enables us to derive an exact relation between the exact marginals on an arbitrary graph with cycles, and the approximations provided by belief propagation, and more broadly, any algorithm that minimizes the Bethe free energy. We also develop bounds on this approximation error, which illuminate the conditions that govern their accuracy. Finally, we show how the reparameterization perspective extends naturally to more structured approximations (e.g., Kikuchi and variants [52, 37]) that operate over higher order cliques.
Graphcover decoding and finitelength analysis of messagepassing iterative decoding of LDPC codes
 IEEE TRANS. INFORM. THEORY
, 2005
"... The goal of the present paper is the derivation of a framework for the finitelength analysis of messagepassing iterative decoding of lowdensity paritycheck codes. To this end we introduce the concept of graphcover decoding. Whereas in maximumlikelihood decoding all codewords in a code are comp ..."
Abstract

Cited by 67 (12 self)
 Add to MetaCart
The goal of the present paper is the derivation of a framework for the finitelength analysis of messagepassing iterative decoding of lowdensity paritycheck codes. To this end we introduce the concept of graphcover decoding. Whereas in maximumlikelihood decoding all codewords in a code are competing to be the best explanation of the received vector, under graphcover decoding all codewords in all finite covers of a Tanner graph representation of the code are competing to be the best explanation. We are interested in graphcover decoding because it is a theoretical tool that can be used to show connections between linear programming decoding and messagepassing iterative decoding. Namely, on the one hand it turns out that graphcover decoding is essentially equivalent to linear programming decoding. On the other hand, because iterative, locally operating decoding algorithms like messagepassing iterative decoding cannot distinguish the underlying Tanner graph from any covering graph, graphcover decoding can serve as a model to explain the behavior of messagepassing iterative decoding. Understanding the behavior of graphcover decoding is tantamount to understanding
TreeBased Reparameterization for Approximate Estimation on Loopy Graphs
 Advances in Neural Information Processing Systems (NIPS
, 2001
"... We present a treebased reparameterization framework that provides a new conceptual view of a large class of iterative algorithms for computing approximate marginals in graphs with cycles. It includes belief propagation (BP), which can be reformulated as a very local form of reparameterization. Mor ..."
Abstract

Cited by 49 (4 self)
 Add to MetaCart
We present a treebased reparameterization framework that provides a new conceptual view of a large class of iterative algorithms for computing approximate marginals in graphs with cycles. It includes belief propagation (BP), which can be reformulated as a very local form of reparameterization. More generally, we consider algorithms that perform exact computations over spanning trees of the full graph. On the practical side, we nd that such tree reparameterization (TRP) algorithms typically converge more quickly than BP with lower cost per iteration; moreover, TRP often converges on problems for which BP fails. The reparameterization perspective also provides theoretical insight into approximate estimation, including a new probabilistic characterization of xed points; and an invariance intrinsic to TRP/BP. These two properties in conjunction enable us to analyze and bound the approximation error that arises in applying these techniques. Our results also have natural extensions to approximations (e.g., Kikuchi) that involve clustering nodes. 1
On the Convergence of Iterative Decoding on Graphs with a Single Cycle
 In Proc. 1998 ISIT
, 1998
"... It is now understood [7, 8] that the turbo decoding algorithm is an instance of a probability propagation algorithm (PPA) on a graph with many cycles. However, PPAtype algorithms are known to give exact results only when the underlying graph is cyclefree. Thus it is important to study the "approxi ..."
Abstract

Cited by 43 (1 self)
 Add to MetaCart
It is now understood [7, 8] that the turbo decoding algorithm is an instance of a probability propagation algorithm (PPA) on a graph with many cycles. However, PPAtype algorithms are known to give exact results only when the underlying graph is cyclefree. Thus it is important to study the "approximate correctness" of PPA on graphs with cycles. In this paper we make a first step by discussing the behavior of an PPA in graphs with a single cycle. This work is directly relevant to the study of iterative decoding of tailbiting codes, whose underlying graph has just one cycle [3], [12]. First, we shall show that for strictly positive local kernels, the iterations of the PPA will always converge to the same fixed point regardless of the scheduling order used. Moreover, the length of the cycle does not play a role in this convergence. Secondly, we shall generalize a result of McEliece and Rodemich [9], by showing that if the hidden variables in the cycle are binaryvalued, a decision based...
The Structure of TailBiting Trellises: Minimality and Basic Principles
 IEEE Trans. Inform. Theory
, 2002
"... Basic structural properties of tailbiting trellises are investigated. We start with rigorous definitions of various types of minimality for tailbiting trellises. ..."
Abstract

Cited by 17 (1 self)
 Add to MetaCart
Basic structural properties of tailbiting trellises are investigated. We start with rigorous definitions of various types of minimality for tailbiting trellises.
Iterative MinSum Decoding of Tailbiting Codes
 In Proc. IEEE Information Theory Workshop
, 1998
"... By invoking a form of the PerronFrobenius theorem for the "minsum" semiring, we obtain a union bound on the performance of iterative decoding of tailbiting codes. This bound shows that for the Gaussian channel, iterative decoding will be optimum, at least for high SNRs, if and only if the minimum ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
By invoking a form of the PerronFrobenius theorem for the "minsum" semiring, we obtain a union bound on the performance of iterative decoding of tailbiting codes. This bound shows that for the Gaussian channel, iterative decoding will be optimum, at least for high SNRs, if and only if the minimum "pseudodistance" of the code is larger than the ordinary minimum distance. I. Introduction Because of the remarkable success of the iterative turbodecoding algorithm [4], many coding researchers have been focussing on the study of other, more easily analyzed, suboptimal iterative decoding algorithms. Perhaps the simplest such algorithm is the iterative decoding of tailbiting codes. In this paper we announce the result that iterative minsum decoding of a tailbiting code will be effective if and only if the minimum "pseudoweight" of the code is strictly greater than its ordinary minimum weight. We are, however, only able to define the pseudoweight for the AWGN channel. II. PerronFrobeni...
Analog TurboNetworks in VLSI: The Next Step in Turbo Decoding and Equalization
 in Proc. Int. Symp. on Turbo Codes and Related Topics
, 2000
"... Turbo decoding is a step towards ana log because it uses softin/softout decoders. The next step is to go fully analog by exchanging extrinsic in formation in continuous time ('flooding'). The component decoders are implemented in analog VLSI (a simple chip exists in 0.25/ra BiCMOS technology) an ..."
Abstract

Cited by 11 (1 self)
 Add to MetaCart
Turbo decoding is a step towards ana log because it uses softin/softout decoders. The next step is to go fully analog by exchanging extrinsic in formation in continuous time ('flooding'). The component decoders are implemented in analog VLSI (a simple chip exists in 0.25/ra BiCMOS technology) and perform trellis or message passing decoding in continuous time while interconnected by an interleaver network. Simulation results for 'turbo' receivers with Hamming, tailbiting convolutional, DECPSK and multipath channel codes have shown that the performance is comparable with digital 'turbo' decoders. The advantage with analog 'turbo' networks is that they operate at much higher speed, have a smaller size and have much less power consumption than their digital counterparts.
LowVoltage CMOS Circuits for Analog Iterative Decoders
 IEEE TRANS. CIRCUITS AND SYSTEMS I: REGULAR PAPERS
, 2006
"... Iterative decoders, including Turbo decoders, provide nearoptimal error protection for various communication channels and storage media. CMOS analog implementations of these decoders offer dramatic savings in complexity and power consumption, compared to digital architectures. Conventional CMOS an ..."
Abstract

Cited by 9 (2 self)
 Add to MetaCart
Iterative decoders, including Turbo decoders, provide nearoptimal error protection for various communication channels and storage media. CMOS analog implementations of these decoders offer dramatic savings in complexity and power consumption, compared to digital architectures. Conventional CMOS analog decoders must have supply voltage greater than 1 V. A new lowvoltage architecture is proposed which reduces the required supply voltage by at least 0.4 V. It is shown that the lowvoltage architecture can be used to implement the general sumproduct algorithm. The lowvoltage analog architecture is then useful for implementing Turbo and lowdensity parity check decoders. The lowvoltage architecture introduces new requirements for signal normalization, which are discussed. Measured results for two fabricated lowvoltage analog decoders are also presented.
TreeBased Reparameterization Framework for Approximate Estimation of Stochastic Processes on Graphs With Cycles
, 2001
"... We present a treebased reparameterization framework for the approximate estimation of stochastic processes on graphs with cycles. This framework provides a new conceptual view of a large class of iterative algorithms for computing approximate marginals in graphs with cycles. Among them is belief pr ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
We present a treebased reparameterization framework for the approximate estimation of stochastic processes on graphs with cycles. This framework provides a new conceptual view of a large class of iterative algorithms for computing approximate marginals in graphs with cycles. Among them is belief propagation (BP), otherwise known as the sumproduct algorithm, which can be reformulated as a very local form of reparameterization. More generally, this class includes algorithms in which updates are more global, and involve performing exact computations over spanning trees of the full graph. On the practical side, we nd that such tree reparameterization (TRP) algorithms typically converge more quickly than belief propagation with equivalent or lower cost per iteration; moreover, TRP often converges on harder problems for which BP fails. The reparameterization perspective also provides new theoretical insights into approximate estimation. In particular, it leads to a novel probabilistic characterization of the set of xed points. We develop the geometry of treebased reparameterization, and use it to develop sucient conditions for convergence in the case of two spanning trees. A fundamental property of reparameterization updates is that they leave invariant the distribution on the full graph. This invariance, in conjunction with our xed point characterization, enables us to derive an exact relation between the true marginals on an arbitrary graph with cycles, and the approximations provided by TRP or BP. We also develop bounds on this approximation error, which illuminate the conditions that govern performance of such techniques. Our results also have natural extensions to approximations (e.g., Kikuchi) that involve clustering nodes. MW was supported in part by a 1967 Fel...
On Dualizing TrellisBased APP Decoding Algorithms
 IEEE Trans. on Communications
, 2002
"... The trellis of a finite Abelian group code is locally (i.e., trellis section by trellis section) related to the trellis of the corresponding dual group code which allows to express the basic operations of the a posteriori probability (APP) decoding algorithm (defined on a single trellis section of t ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
The trellis of a finite Abelian group code is locally (i.e., trellis section by trellis section) related to the trellis of the corresponding dual group code which allows to express the basic operations of the a posteriori probability (APP) decoding algorithm (defined on a single trellis section of the primal trellis) in terms of the corresponding dual trellis section. Using this local approach, any algorithm employing the same type of operations as the APP algorithm can, thus, be dualized, even if the global dual code does not exist (e.g., nongroup codes represented by a group trellis). Given this, the complexity advantage of the dual approach for highrate codes can be generalized to a broader class of APP decoding algorithms, including suboptimum algorithms approximating the true APP, which may be more attractive in practical applications due to their reduced complexity. Moreover, the local approach opens the way for mixed approaches where the operations of the APP algorithm are not exclusively performed on the primal or dual trellis. This is inevitable if the code does not possess a trellis consisting solely of group trellis sections as, e.g., for certain terminated group or ring codes. The complexity reduction offered by applying dualization is evaluated. As examples, we give a dual implementation of a suboptimum APP decoding algorithm for tailbiting convolutional codes, as well as dual implementations of APP algorithms of the slidingwindow type. Moreover, we evaluate their performance for decoding usual tailbiting codes or convolutional codes, respectively, as well as their performance as component decoders in iteratively decoded parallel concatenated schemes.