Results 1  10
of
19
Factor Graphs and the SumProduct Algorithm
 IEEE TRANSACTIONS ON INFORMATION THEORY
, 1998
"... A factor graph is a bipartite graph that expresses how a "global" function of many variables factors into a product of "local" functions. Factor graphs subsume many other graphical models including Bayesian networks, Markov random fields, and Tanner graphs. Following one simple c ..."
Abstract

Cited by 1791 (69 self)
 Add to MetaCart
A factor graph is a bipartite graph that expresses how a "global" function of many variables factors into a product of "local" functions. Factor graphs subsume many other graphical models including Bayesian networks, Markov random fields, and Tanner graphs. Following one simple computational rule, the sumproduct algorithm operates in factor graphs to computeeither exactly or approximatelyvarious marginal functions by distributed messagepassing in the graph. A wide variety of algorithms developed in artificial intelligence, signal processing, and digital communications can be derived as specific instances of the sumproduct algorithm, including the forward/backward algorithm, the Viterbi algorithm, the iterative "turbo" decoding algorithm, Pearl's belief propagation algorithm for Bayesian networks, the Kalman filter, and certain fast Fourier transform algorithms.
Improved lowdensity paritycheck codes using irregular graphs
 IEEE Trans. Inform. Theory
, 2001
"... Abstract—We construct new families of errorcorrecting codes based on Gallager’s lowdensity paritycheck codes. We improve on Gallager’s results by introducing irregular paritycheck matrices and a new rigorous analysis of harddecision decoding of these codes. We also provide efficient methods for ..."
Abstract

Cited by 223 (15 self)
 Add to MetaCart
Abstract—We construct new families of errorcorrecting codes based on Gallager’s lowdensity paritycheck codes. We improve on Gallager’s results by introducing irregular paritycheck matrices and a new rigorous analysis of harddecision decoding of these codes. We also provide efficient methods for finding good irregular structures for such decoding algorithms. Our rigorous analysis based on martingales, our methodology for constructing good irregular codes, and the demonstration that irregular structure improves performance constitute key points of our contribution. We also consider irregular codes under belief propagation. We report the results of experiments testing the efficacy of irregular codes on both binarysymmetric and Gaussian channels. For example, using belief propagation, for rate I R codes on 16 000 bits over a binarysymmetric channel, previous lowdensity paritycheck codes can correct up to approximately 16 % errors, while our codes correct over 17%. In some cases our results come very close to reported results for turbo codes, suggesting that variations of irregular low density paritycheck codes may be able to match or beat turbo code performance. Index Terms—Belief propagation, concentration theorem, Gallager codes, irregular codes, lowdensity paritycheck codes.
Iterative Decoding of Compound Codes by Probability Propagation in Graphical Models
 IEEE J. Sel. Areas Comm
, 1998
"... Abstract—We present a unified graphical model framework for describing compound codes and deriving iterative decoding algorithms. After reviewing a variety of graphical models (Markov random fields, Tanner graphs, and Bayesian networks), we derive a general distributed marginalization algorithm for ..."
Abstract

Cited by 139 (12 self)
 Add to MetaCart
(Show Context)
Abstract—We present a unified graphical model framework for describing compound codes and deriving iterative decoding algorithms. After reviewing a variety of graphical models (Markov random fields, Tanner graphs, and Bayesian networks), we derive a general distributed marginalization algorithm for functions described by factor graphs. From this general algorithm, Pearl’s belief propagation algorithm is easily derived as a special case. We point out that recently developed iterative decoding algorithms for various codes, including “turbo decoding ” of parallelconcatenated convolutional codes, may be viewed as probability propagation in a graphical model of the code. We focus on Bayesian network descriptions of codes, which give a natural input/state/output/channel description of a code and channel, and we indicate how iterative decoders can be developed for paralleland serially concatenated coding systems, product codes, and lowdensity paritycheck codes. Index Terms — Concatenated coding, decoding, graph theory, iterative methods, product codes.
Analysis of Low Density Codes and Improved Designs Using Irregular Graphs
, 1998
"... In [6], Gallager introduces a family of codes based on sparse bipartite graphs, which he calls lowdensity paritycheck codes. He suggests a natural decoding algorithm for these codes, and proves a good bound on the fraction of errors that can be corrected. As the codes that Gallager builds are deri ..."
Abstract

Cited by 88 (12 self)
 Add to MetaCart
In [6], Gallager introduces a family of codes based on sparse bipartite graphs, which he calls lowdensity paritycheck codes. He suggests a natural decoding algorithm for these codes, and proves a good bound on the fraction of errors that can be corrected. As the codes that Gallager builds are derived from regular graphs, we refer to them as regular codes. Following the general approach introduced in [7] for the design and analysis of erasure codes, we consider errorcorrecting codes based on random irregular bipartite graphs, which we call irregular codes. We introduce tools based on linear programming for designing linear time irregular codes with better errorcorrecting capabilities than possible with regular codes. For example, the decoding algorithm for the rate 1/2 regular codes of Gallager can provably correct up to 5.17% errors asymptotically, whereas we have found irregular codes for which our decoding algorithm can provably correct up to 6.27% errors asymptotically. We incl...
Belief Propagation and Revision in Networks with Loops
, 1997
"... Local belief propagation rules of the sort proposed by Pearl (1988) are guaranteed to converge to the optimal beliefs for singly connected networks. Recently, a number of researchers have empirically demonstrated good performance of these same algorithms on networks with loops, but a theoretical und ..."
Abstract

Cited by 77 (5 self)
 Add to MetaCart
Local belief propagation rules of the sort proposed by Pearl (1988) are guaranteed to converge to the optimal beliefs for singly connected networks. Recently, a number of researchers have empirically demonstrated good performance of these same algorithms on networks with loops, but a theoretical understanding of this performance has yet to be achieved. Here we lay a foundation for an understanding of belief propagation in networks with loops. For networks with a single loop, we derive an analytical relationship between the steady state beliefs in the loopy network and the true posterior probability. Using this relationship we show a category of networks for which the MAP estimate obtained by belief update and by belief revision can be proven to be optimal (although the beliefs will be incorrect). We show how nodes can use local information in the messages they receive in order to correct the steady state beliefs. Furthermore we prove that for all networks with a single loop, the MAP es...
A factor graph framework for semantic video indexing
 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
, 2002
"... ... most challenging research issues in video data management. To go beyond lowlevel similarity and access video data content by semantics, we need to bridge the gap between the lowlevel representation and highlevel semantics. This is a difficult multimedia understanding problem. We formulate thi ..."
Abstract

Cited by 61 (6 self)
 Add to MetaCart
(Show Context)
... most challenging research issues in video data management. To go beyond lowlevel similarity and access video data content by semantics, we need to bridge the gap between the lowlevel representation and highlevel semantics. This is a difficult multimedia understanding problem. We formulate this problem as a probabilistic patternrecognition problem for modeling semantics in terms of concepts and context. To map lowlevel features to highlevel semantics, we propose probabilistic multimedia objects (multijects). Examples of multijects in movies include explosion, mountain, beach, outdoor, music, etc. Semantic concepts in videos interact and appear in context. To model this interaction explicitly, we propose a network of multijects (multinet). To model the multinet computationally, we propose a factor graph framework which can enforce spatiotemporal constraints. Using probabilistic models for multijects, rocks, sky, snow, waterbody, and forestry/greenery, and using a factor graph as the multinet, we demonstrate the application of this framework to semantic video indexing. We demonstrate how detection performance can be significantly improved using the multinet to take interconceptual relationships into account. Our experiments using a large video database consisting of clips from several movies and based on a set of five semantic concepts reveal a significant improvement in detection performance by over 22%. We also show how the multinet is extended to take temporal correlation into account. By constructing a dynamic multinet, we show that the detection performance is further enhanced by as much as 12%. With this framework, we show how keywordbased query and semantic filtering is possible for a predetermined set of concepts.
Extracting Semantics from Audiovisual Content: The Final Frontier in Multimedia Retrieval
 IEEE TRANSACTIONS ON NEURAL NETWORKS
, 2002
"... Multimedia understanding is a fast emerging interdisciplinary research area. There is tremendous potential for effective use of multimedia content through intelligent analysis. Diverse application areas are increasingly relying on multimedia understanding systems. Advances in multimedia understandi ..."
Abstract

Cited by 48 (0 self)
 Add to MetaCart
(Show Context)
Multimedia understanding is a fast emerging interdisciplinary research area. There is tremendous potential for effective use of multimedia content through intelligent analysis. Diverse application areas are increasingly relying on multimedia understanding systems. Advances in multimedia understanding are related directly to advances in signal processing, computer vision, pattern recognition, multimedia databases, and smart sensors. We review the stateoftheart techniques in multimedia retrieval. In particular we discuss how multimedia retrieval can be viewed as a pattern recognition problem. We discuss, how reliance on powerful pattern recognition and machine learning techniques is increasing in the field of multimedia retrieval. We review stateoftheart multimedia understanding systems with particular emphasis on a system for semantic video indexing centered around multijects and multinets. We discuss how semantic retrieval is centered around concepts and context and also discuss various mechanisms for modeling concepts and context.
An Analysis of Belief Propagation on the Turbo Decoding Graph with Gaussian Densities
 IEEE Transactions on Information Theory
, 2000
"... Motivated by its success in decoding turbo codes, we provide an analysis of the belief propagation algorithm on the turbo decoding graph with Gaussian densities. In this context, we are able to show that, under certain conditions, the algorithm converges and that  somewhat surprisingly  though t ..."
Abstract

Cited by 47 (6 self)
 Add to MetaCart
(Show Context)
Motivated by its success in decoding turbo codes, we provide an analysis of the belief propagation algorithm on the turbo decoding graph with Gaussian densities. In this context, we are able to show that, under certain conditions, the algorithm converges and that  somewhat surprisingly  though the density generated by belief propagation may di#er significantly from the desired posterior density, the means of these two densities coincide. Since computation of posterior distributions is tractable when densities are Gaussian, use of belief propagation in such a setting may appear unwarranted. Indeed, our primary motivation for studying belief propagation in this context stems from a desire to enhance our understanding of the algorithm's dynamics in nonGaussian setting, and to gain insights into its excellent performance in turbo codes. Nevertheless, even when the densities are Gaussian, belief propagation may sometimes provide a more e#cient alternative to traditional inference metho...
Algorithmic Complexity in Coding Theory and the Minimum Distance Problem
, 1997
"... We start with an overview of algorithmiccomplexity problems in coding theory We then show that the problem of computing the minimum distance of a binary linear code is NPhard, and the corresponding decision problem is NPcomplete. This constitutes a proof of the conjecture Bedekamp, McEliece, van T ..."
Abstract

Cited by 44 (2 self)
 Add to MetaCart
We start with an overview of algorithmiccomplexity problems in coding theory We then show that the problem of computing the minimum distance of a binary linear code is NPhard, and the corresponding decision problem is NPcomplete. This constitutes a proof of the conjecture Bedekamp, McEliece, van Tilborg, dating back to 1978. Extensions and applications of this result to other problems in coding theory are discussed.
Very Loopy Belief Propagation for Unwrapping Phase Images
, 2001
"... Since the discovery that the best errorcorrecting decoding algorithm can be viewed as belief propagation in a cyclebound graph, researchers have been trying to determine under what circumstances "loopy belief propagation" is eective for probabilistic inference. Despite several ..."
Abstract

Cited by 33 (4 self)
 Add to MetaCart
Since the discovery that the best errorcorrecting decoding algorithm can be viewed as belief propagation in a cyclebound graph, researchers have been trying to determine under what circumstances "loopy belief propagation" is eective for probabilistic inference. Despite several