Abstract:
Algorithms that must deal with complicated global functions of many variables often exploit the manner in which the given functions factor as a product of "local" functions, each of which depends on a subset of the variables. Such a factorization can be visualized with a bipartite graph that we call a factor graph. In this tutorial paper, we present a generic message-passing algorithm, the sum-product algorithm, that operates in a factor graph. Following a single, simple computational rule, the sum-product algorithm computes---either exactly or approximately---various marginal functions derived from the global function. A wide variety of algorithms developed in artificial intelligence, signal processing, and digital communications can be derived as specific instances of the sum-product algorithm, including the forward/backward algorithm, the Viterbi algorithm, the iterative "turbo" decoding algorithm, Pearl's belief propagation algorithm for Bayesian networks, the Kalman filter, and certain fast Fourier transform (FFT) algorithms.
Citations
|
7271
|
Computers and Intractability - A Guide to the Theory of NP-Completeness
– Garey, Johnson
- 1979
|
|
936
|
Local Computations with Probabilities on Graphical Structures and Their Applications to Expert Systems
– Lauritzen, Spigelholter
- 1988
|
|
838
|
Near Shannon limit error-correcting coding and decoding
– Berrou, Glavieux, et al.
- 1993
|
|
741
|
Optimal decoding of linear codes for minimizing symbol error rate
– Bahl, Cocke, et al.
- 1974
|
|
619
|
Introduction to Bayesian Networks
– Jensen
- 1996
|
|
410
|
Low-Density Parity-Check Codes
– Gallager
- 1963
|
|
373
|
Concrete mathematics
– Graham, Knuth, et al.
- 1991
|
|
372
|
Optimal Filtering
– Anderson, Moore
- 1979
|
|
304
|
Iterative decoding of binary block and convolutional codes
– Hagenauer, Offer, et al.
- 1996
|
|
221
|
Learning and relearning in Boltzmann machines
– Hinton, TJ
- 1986
|
|
216
|
Statistical inference for probabilistic functions of finite state Markov chains, Annals of Mathematical Statistics 41
– Baum, Petrie
|
|
175
|
Graphical Models for Machine Learning and Digital Communication
– Frey
- 1998
|
|
171
|
The wake-sleep algorithm for unsupervised neural networks
– Hinton, Dayan, et al.
- 1995
|
|
149
|
The Helmholtz machine
– Dayan, Hinton, et al.
- 1995
|
|
147
|
Nonserial Dynamic Programming
– Bertele, Brioschi
- 1972
|
|
91
|
Coding theorems for ’turbo-like’ codes
– Divsalar, Jin, et al.
- 1998
|
|
81
|
Iterative decoding of compound codes by probability propagation in graphical models
– Kschischang, Frey
- 1998
|
|
70
|
Markov Random Fields and their Applications
– Kindermann, Snell
- 1980
|
|
54
|
Good codes based on very sparse matrices
– MacKay, Neal
- 1995
|
|
27
|
Iterative decoding of serially concatenated convolutional codes
– Benedetto, Montorsi
- 1996
|
|
27
|
Local computation with valuations from a commutative semigroup
– LAURITZEN, V
- 1997
|
|
15
|
Variational learning in nonlinear Gaussian belief networks
– Frey, Hinton
- 1999
|
|
15
|
Probability propagation and iterative decoding
– Frey, Kschischang
- 1996
|
|
11
|
An introduction to spatial point processes and Markov random fields
– Isham
- 1981
|
|
8
|
A general algorithm for distributing information on a graph
– Aji, McEliece
- 1997
|
|
8
|
On iterative decoding and the two-way algorithm
– Forney
- 1997
|
|
5
|
On iterative decoding and the two-way algorithm
– Jr
- 1997
|
|
4
|
A simple algorithm that discovers efficient perceptual codes
– Frey, Dayan, et al.
- 1997
|
|
2
|
generalized distributive law
– “The
- 1997
|
|
2
|
Interleaver properties and their applications to the trellis complexity analysis of turbo codes
– Garello, Montorsi, et al.
- 2001
|
|
1
|
The generalized distributive law," preprint available on-line from http://www.systems.caltech.edu/EE/faculty/rjm
– Aji, McEliece
- 1998
|
|
1
|
on graphs: Normal realizations
– “Codes
- 2001
|
