## Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms (2005)

### Cached

### Download Links

- [www3.cs.pitt.edu]
- [people.cs.pitt.edu]
- [people.cs.pitt.edu]
- [www.cs.utsa.edu]
- [www.merl.com]
- [www.cs.utsa.edu]
- [cba.mit.edu]
- [www.ics.uci.edu]
- [www.cs.duke.edu]
- [www.cs.duke.edu]
- [www.cs.huji.ac.il]
- [www.merl.com]
- [www.merl.com]
- [www.merl.com]
- [www.merl.com]
- [www.merl.com]
- [www.merl.com]
- [www.merl.com]
- [carlit.toulouse.inra.fr]
- [www.merl.com]
- [www.cs.princeton.edu]
- [www.gatsby.ucl.ac.uk]
- [www.gatsby.ucl.ac.uk]
- [www.stanford.edu]

Venue: | IEEE Transactions on Information Theory |

Citations: | 413 - 12 self |

### BibTeX

@ARTICLE{Yedidia05constructingfree,

author = {Jonathan S. Yedidia and William T. Freeman and Yair Weiss},

title = {Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms},

journal = {IEEE Transactions on Information Theory},

year = {2005},

volume = {51},

pages = {2282--2312}

}

### Years of Citing Articles

### OpenURL

### Abstract

Important inference problems in statistical physics, computer vision, error-correcting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree, but only approximate when the factor graph has cycles. We show that BP fixed points correspond to the stationary points of the Bethe approximation of the free energy for a factor graph. We explain how to obtain regionbased free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms. We emphasize the conditions a free energy approximation must satisfy in order to be a “valid ” or “maxent-normal ” approximation. We describe the relationship between four different methods that can be used to generate valid approximations: the “Bethe method, ” the “junction graph method, ” the “cluster variation method, ” and the “region graph method.” Finally, we explain how to tell whether a region-based approximation, and its corresponding GBP algorithm, is likely to be accurate, and describe empirical results showing that GBP can significantly outperform BP.

### Citations

8564 |
Elements of Information Theory
- Cover, Thomas
- 2003
(Show Context)
Citation Context ... ��� ���©��� ������� �������� � (17) � ����� � ����� (18) �������� � ����� ��� � � ��=-=��� � � � is the ����� � Kullback-Leibler ����� � divergence between and . Since there exists a � �������� � theorem [30] that is always nonnegative and is zero if ����� ��� ����� � and only if , we see th-=-at � ��� ������� ��� , with equality precisely when ����� ��� ����� � . � ��� �����sMinimizing the Gibbs free ����� ... |

7048 |
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
- Pearl
- 1988
(Show Context)
Citation Context ...5], Gallager’s sum-product algorithm for decoding lowdensity parity check codes [6], the “turbo-decoding” algorithm [7], [8], Pearl’s “belief propagation” algorithm for inference on Bayesi=-=an networks [9], the “K-=-alman filter” for signal processing [10], [11], and the “transfer matrix” approach in statistical mechanics [12]. MERL Cambridge Research Lab, 201 Broadway, 8th Floor, Cambridge £ MA 02139. ¤ ... |

4269 | A tutorial on hidden Markov models and selected applications in speech recognition
- Rabiner
- 1989
(Show Context)
Citation Context ...quivalent or very closely-related message-passing algorithms have now been independently invented many times. They are well-known by names like the forward-backward algorithm for Hidden Markov Models =-=[3], the Viterb-=-i algorithm [4], [5], Gallager’s sum-product algorithm for decoding lowdensity parity check codes [6], the “turbo-decoding” algorithm [7], [8], Pearl’s “belief propagation” algorithm for i... |

2108 |
A new approach to linear filtering and prediction problems
- Kalman
- 1960
(Show Context)
Citation Context ...ng lowdensity parity check codes [6], the “turbo-decoding” algorithm [7], [8], Pearl’s “belief propagation” algorithm for inference on Bayesian networks [9], the “Kalman filter” for sign=-=al processing [10], [11]-=-, and the “transfer matrix” approach in statistical mechanics [12]. MERL Cambridge Research Lab, 201 Broadway, 8th Floor, Cambridge £ MA 02139. ¤ yedidia@merl.com Electrical Engineering and Comp... |

1476 |
Exactly Solved Models in Statistical Mechanics
- Baxter
- 1982
(Show Context)
Citation Context ..., [8], Pearl’s “belief propagation” algorithm for inference on Bayesian networks [9], the “Kalman filter” for signal processing [10], [11], and the “transfer matrix” approach in statisti=-=cal mechanics [12]. -=-MERL Cambridge Research Lab, 201 Broadway, 8th Floor, Cambridge £ MA 02139. ¤ yedidia@merl.com Electrical Engineering and Computer Science, MIT Artificial Intelligence Laboratory, NE43a, Cambridge M... |

1304 | Near Shannon limit error-correcting coding and decoding: turbo code
- Berrou, Glavieux, et al.
(Show Context)
Citation Context ...orward-backward algorithm for Hidden Markov Models [3], the Viterbi algorithm [4], [5], Gallager’s sum-product algorithm for decoding lowdensity parity check codes [6], the “turbo-decoding” algo=-=rithm [7], [8], Pearl��-=-�s “belief propagation” algorithm for inference on Bayesian networks [9], the “Kalman filter” for signal processing [10], [11], and the “transfer matrix” approach in statistical mechanics ... |

1166 | Factor graphs and the sum-product algorithm
- Kschischang, Frey, et al.
- 2001
(Show Context)
Citation Context ...ressed for other graphical models without difficulty. Using factor graphs has certain practical advantages–in particular we can refer the neophyte reader to the excellent review by Kschischang et.al=-=. [20]-=-. That review explains the equivalence to factor graphs of other graphical models such as Bayesian networks, Tanner graphs for error-correcting codes, or pair-wise Let be a set of discrete-valued rand... |

1163 |
Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm
- Viterbi
- 1967
(Show Context)
Citation Context ...elated message-passing algorithms have now been independently invented many times. They are well-known by names like the forward-backward algorithm for Hidden Markov Models [3], the Viterbi algorithm =-=[4], [5], Galla-=-ger’s sum-product algorithm for decoding lowdensity parity check codes [6], the “turbo-decoding” algorithm [7], [8], Pearl’s “belief propagation” algorithm for inference on Bayesian networ... |

839 |
Enumerative combinatorics
- Stanley
- 1997
(Show Context)
Citation Context ...ationship, a region graph must be a directed acyclic graph, in the sense that the arrows cannot loop around. A region graph is closely related to the Hasse diagram for a partially ordered set,orposet =-=[53]-=-, if we consider our regions to be organized into a poset, with the ordering relationship between the regions to be given by the ancestor–descendant relationship [30], [31]. There are, however, some d... |

831 | An introduction to variational methods for graphical models
- Jordan, Ghahramani, et al.
- 1999
(Show Context)
Citation Context ... . Instead of a factorized form, one might consider other more � complicated forms for which still lead to tractable approx����� imations. This is the idea behind the “structured mean-fi=-=eld” approach [31]. We will no-=-t follow that path, and will instead describe a quite ����� � different approach to approximating in the next section; one which underlies the BP algorithm. IV. REGION-BASED FREE ENERGY AP... |

746 | The Viterbi algorithm
- Forney, D
- 1973
(Show Context)
Citation Context ...d message-passing algorithms have now been independently invented many times. They are well-known by names like the forward-backward algorithm for Hidden Markov Models [3], the Viterbi algorithm [4], =-=[5], Gallager��-=-�s sum-product algorithm for decoding lowdensity parity check codes [6], the “turbo-decoding” algorithm [7], [8], Pearl’s “belief propagation” algorithm for inference on Bayesian networks [9... |

745 |
Nonlinear Programming
- Bertsekas
- 2004
(Show Context)
Citation Context .... A. Review of Lagrangian Formalism We first briefly review some necessary background about the Lagrangian formalism for constrained optimization. An excellent textbook containing more information is =-=[44]-=-. Consider a function of variables , where the variables may be subject to equality constraint(s) (written as ) and inequality constraint(s) (written as ). We will assume throughout that the equality ... |

714 |
Applied Optimal Estimation
- Gelb, editor
- 1974
(Show Context)
Citation Context ...il). Communicated by A. Kavčić, Associate Editor for Detection and Estimation. Digital Object Identifier 10.1109/TIT.2005.850085 Bayesian networks [9], the “Kalman filter” for signal processing [10], =-=[11]-=-, and the “transfer matrix” approach in statistical mechanics [12]. In this list of “standard” belief propagation (BP) algorithms, we have blurred a distinction between two different objectives that o... |

611 | Learning in Graphical Models
- Jordan, editor
- 1999
(Show Context)
Citation Context ...obabilistic inference using graphical models are important in a wide variety of disciplines, including statistical physics, signal processing, artificial intelligence, and digital communications [1], =-=[2]-=-. Message-passing algorithms are a practical and powerful way to solve such problems. The centrality of such problems and the utility of messagepassing algorithms for solving them is an explanation fo... |

476 |
Statistical Mechanics
- Huang
- 2003
(Show Context)
Citation Context ...ere ��� � � � �sis the temperature. Many systems, for example Ising fer� ¢ romagnets, will have different numbers of solutions above or below a critical � � temperature within the=-= Bethe approximation [34]. � ��-=-� Above , the constrained free energy is convex and has a unique stationary point, while � � below , there are multiple stationary points. Using this equivalence it is easy to define small factor ... |

465 | Loopy belief propagation for approximate inference: An empirical study
- Murphy, Weiss, et al.
- 1999
(Show Context)
Citation Context ...phical models with cycles. Nevertheless, in such cases there are no guarantees, and sometimes the results are quite poor, or the algorithm fails to give any result at all because it does not converge =-=[14]-=-. Two major goals of this paper are to explain why the standard BP algorithm often works so well even for graphical models with cycles, and to use that understanding to develop improved algorithms for... |

427 | Graphical models, exponential families, and variational inference
- Wainwright, Jordan
- 2003
(Show Context)
Citation Context ...osed new variational inference techniques, closely related to our region-based approximations, but differentiated by a requirement that the set of beliefs used must be marginals of some global belief =-=[43]. Th-=-ey call the set of beliefs realizable from a global belief the “marginal polytope.” B. Negative Entropies Because some of the terms in the Bethe entropy have a sign that is flipped from the normal... |

397 | Generalized belief propagation
- Yedidia, Freeman, et al.
- 2000
(Show Context)
Citation Context ...r. We did this in an attempt der to ensure that the resulting approximations are accurate. to help the reader grasp the fundamental concepts behind our In our original work introducing GBP algorithms =-=[17]-=-, we fo- work and not lose sight of the forest because of all the trees. cused on a sub-class of GBP algorithms that were equivalent The appendices describe a variety of other methods to generate to f... |

308 | Turbo decoding as an instance of Pearl’s ”belief propagation” algorithm
- McEliece, MacKay, et al.
- 1998
(Show Context)
Citation Context ...d-backward algorithm for Hidden Markov Models [3], the Viterbi algorithm [4], [5], Gallager’s sum-product algorithm for decoding lowdensity parity check codes [6], the “turbo-decoding” algorithm=-= [7], [8], Pearl’s ��-=-�belief propagation” algorithm for inference on Bayesian networks [9], the “Kalman filter” for signal processing [10], [11], and the “transfer matrix” approach in statistical mechanics [12].... |

308 |
Expectation Propagation for Approximate Bayesian Inference
- Minka
- 2001
(Show Context)
Citation Context ...ormulations of the standard BP algorithm provide different insights, and we refer the interested reader to a number of important recent papers that exploit alternative views of the BP algorithm [21], =-=[22]-=-, [23], [24], [25], [26]. After our original work which introduced region-based free energies and GBP algorithms based on the cluster variation method, Aji and McEliece introduced a class of free ener... |

298 |
Understanding belief propagation and its generalizations
- Yedidia, Freeman, et al.
- 2003
(Show Context)
Citation Context ...Eliece and Yildirim in [30]. We have also previously released a number of technical reports [33], [34], [35] that are largely superseded by this paper, as well as a somewhat more popular introduction =-=[36]-=-. The outline for the rest of the paper is as follows. In section II, we review and introduce our notation for factor graphs and the standard BP algorithm. In sections III and IV, we introduce and exp... |

293 |
The generalized distributive law
- Aji, McEliece
- 2000
(Show Context)
Citation Context ...of the standard BP algorithm provide different insights, and we refer the interested reader to a number of important recent papers that exploit alternative views of the BP algorithm [21], [22], [23], =-=[24]-=-, [25], [26]. After our original work which introduced region-based free energies and GBP algorithms based on the cluster variation method, Aji and McEliece introduced a class of free energy approxima... |

264 |
A family of algorithms for approximate Bayesian inference
- Minka
- 2001
(Show Context)
Citation Context ...ave been a number of other recent papers that have tried to explain, reformulate, or generalize the standard belief propagation algorithm in a variety of ways. We point the interested reader to [22], =-=[23]-=-, [24], [25], [26], [27], [28]. After our original work which introduced region-based free energies and GBP algorithms based on the cluster variation method, other works appeared which explored parall... |

256 |
Graphical Models for Machine Learning and Digital Communication
- Frey
- 1998
(Show Context)
Citation Context ...ng probabilistic inference using graphical models are important in a wide variety of disciplines, including statistical physics, signal processing, artificial intelligence, and digital communications =-=[1]-=-, [2]. Message-passing algorithms are a practical and powerful way to solve such problems. The centrality of such problems and the utility of messagepassing algorithms for solving them is an explanati... |

176 | Correctness of local probability propagation in graphical models with loops
- Weiss
- 2000
(Show Context)
Citation Context ...the Bethe approximation. In graphs with no more than a single cycle, it was � known � that� if all (� ��� ����� factors are � strictly � positive for all and ), then there =-=was a unique BP fixed point.[33]-=- For general graphs, we can use the equivalence established above to answer a question about the uniqueness of stationary points for the Bethe free energy. The issue of the number of stationary points... |

154 | A new class of upper bounds on the log partition function
- Wainwright, Jaakkola, et al.
- 2005
(Show Context)
Citation Context ...numbers to differ from those given in the Bethe approximation, is one way of deriving the “fractional belief propagation algorithm” [40] and the essentially equivalent “convexified Bethe free en=-=ergy” [41]-=- approximation. In this paper, we will always assume just one set of counting numbers. In fact, not all region-based approximations to the variational free energy are equally good. At this point, we i... |

129 |
The Viterbi algorithm
- Jr
- 1973
(Show Context)
Citation Context ...d message-passing algorithms have now been independently invented many times. They are well known by names like the forward–backward algorithm for hidden Markov models [3], the Viterbi algorithm [4], =-=[5]-=-, Gallager’s sum–product algorithm for decoding lowdensity parity check codes [6], the “turbo-decoding” algorithm [7], [8], Pearl’s “belief propagation” algorithm for inference on Manuscript received ... |

122 |
Codes on Graphs: Normal Realizations
- Forney
- 2001
(Show Context)
Citation Context ...ther formulations of the standard BP algorithm provide different insights, and we refer the interested reader to a number of important recent papers that exploit alternative views of the BP algorithm =-=[21]-=-, [22], [23], [24], [25], [26]. After our original work which introduced region-based free energies and GBP algorithms based on the cluster variation method, Aji and McEliece introduced a class of fre... |

119 |
Low-density parity check codes
- Gallager
- 1963
(Show Context)
Citation Context ...y are well-known by names like the forward-backward algorithm for Hidden Markov Models [3], the Viterbi algorithm [4], [5], Gallager’s sum-product algorithm for decoding lowdensity parity check code=-=s [6], the “turbo-d-=-ecoding” algorithm [7], [8], Pearl’s “belief propagation” algorithm for inference on Bayesian networks [9], the “Kalman filter” for signal processing [10], [11], and the “transfer matrix... |

108 | CCCP algorithms to minimize the Bethe and Kikuchi free energies: Convergent alternatives to belief propagation
- Yuille
- 2002
(Show Context)
Citation Context ...t the beliefs are in a feasible set. Based on the equivalence, first noted in our earlier work [17], others have recently devised algorithms that directly minimize the free energy on the feasible set =-=[35]-=-, [36], [37]. Such free energy minimizations are somewhat slower than the BP algorithm, but they are guaranteed to converge. VI. THE REGION GRAPH METHOD We now introduce region graphs, which are centr... |

101 | Tree-based reparameterization framework for analysis of sum-product and related algorithms
- Wainwright, Jaakkola, et al.
- 2003
(Show Context)
Citation Context ...en a number of other recent papers that have tried to explain, reformulate, or generalize the standard belief propagation algorithm in a variety of ways. We point the interested reader to [22], [23], =-=[24]-=-, [25], [26], [27], [28]. After our original work which introduced region-based free energies and GBP algorithms based on the cluster variation method, other works appeared which explored parallel ide... |

75 | Loopy belief propagation and Gibbs measures
- Tatikonda, Jordan
- 2002
(Show Context)
Citation Context ...fs, but at one of the fixed points, the beliefs are biased towards the first binary state, while at the other fixed point, the beliefs are biased towards the second binary state. Tatikonda and Jordan =-=[47]-=- have explored the question of uniqueness of BP fixed points in detail. They used the connection to the Bethe free energy to obtain a set of sufficient conditions on the strength of the factors fa(xa)... |

70 |
Random k-satisfiability problem: From an analytic solution to an efficient algorithm
- Mézard, Zecchina
- 2002
(Show Context)
Citation Context ...ard BP algorithm provide different insights, and we refer the interested reader to a number of important recent papers that exploit alternative views of the BP algorithm [21], [22], [23], [24], [25], =-=[26]-=-. After our original work which introduced region-based free energies and GBP algorithms based on the cluster variation method, Aji and McEliece introduced a class of free energy approximations and GB... |

70 | Bethe free energy, Kikuchi approximations, and belief propagation algorithms
- Yedidia, Freeman, et al.
- 2001
(Show Context)
Citation Context ...raphs [29]. We also recommend the elegant exposition of generalized belief propagation presented by McEliece and Yildirim in [30]. We have also previously released a number of technical reports [33], =-=[34]-=-, [35] that are largely superseded by this paper, as well as a somewhat more popular introduction [36]. The outline for the rest of the paper is as follows. In section II, we review and introduce our ... |

58 |
A theory of cooperative phenomena
- Kikuchi
- 1951
(Show Context)
Citation Context ...re actually a variety of ways to define GBP alenergies were introduced long ago in the physics literature by gorithms for any given region graph, all of which have identical by Bethe [15] and Kikuchi =-=[16]-=-. For the important special case fixed points. We focus on one particular type of GBP algorithm, of the standard BP algorithm, we show that its fixed points are which we call the parent-to-child algor... |

51 |
Solvable model of a spin-glass,” Phys
- Sherrington, Kirkpatrick
- 1975
(Show Context)
Citation Context ...riable nodes, where every pair of nodes is connected by a factor. (A version of this factor graph with random factors is known in the physics literature as the Sherrington–Kirpatrick Ising spin glass =-=[54]-=-.) Now take, as the regions to include in the region graph, every triplet of nodes (and all three factors that connect them), every pair of nodes (and the factor that connects them), and every single ... |

49 | Tree-based reparameterization for approximate estimation on graphs with cycles
- Wainwright, Jaakkola, et al.
- 2001
(Show Context)
Citation Context ...tions of the standard BP algorithm provide different insights, and we refer the interested reader to a number of important recent papers that exploit alternative views of the BP algorithm [21], [22], =-=[23]-=-, [24], [25], [26]. After our original work which introduced region-based free energies and GBP algorithms based on the cluster variation method, Aji and McEliece introduced a class of free energy app... |

46 | Stable fixed points of loopy belief propagation are minima of the Bethe free energy
- Heskes
- 2003
(Show Context)
Citation Context ...ld be BP fixed points. To complete the general picture of the relation between BP fixed points and the stationary points of the constrained Bethe free energy, we refer the reader to a paper by Heskes =-=[52]-=-, which argues that stable BP fixed points must be local minima of the constrained Bethe free energy, but gives a counter-example that shows that the converse is not true. VII. THE REGION GRAPH METHOD... |

45 | Belief Optimization for Binary Networks: A stable Alternative to Loopy Belief Propagation
- Teh, Welling
- 2001
(Show Context)
Citation Context ... fixed points ane Bethe free energy stationary points, first noted in our earlier work [17], others have devised algorithms that directly minimize the free energy on the feasible set of beliefs [49], =-=[50]-=-, [51]. Such free energy minimizations are somewhat slower than the BP algorithm, but they are guaranteed to converge. D. Factor Graphs Containing Hard Constraints We now return to consider the more g... |

44 |
Tree-structured approximations by expectation propagation
- Minka, Qi
- 2003
(Show Context)
Citation Context ...will disfavor the (correct) uniform distribution. It is therefore no surprise that other researchers have noticed that this approximation gives poor results for the Sherrington-Kirpatrick model [51], =-=[55]-=-. B. Example of an Approximation that is Maxent-Normal Fortunately, it is not too hard to find examples of region graph approximations that are maxent-normal, besides those based on the Bethe approxim... |

43 |
Statistical theory of superlattices
- Bethe
- 1935
(Show Context)
Citation Context ...to another free energy that can be justified by a rigorous vari1sational principle. The first specialized examples of such free energies were introduced long ago in the physics literature by by Bethe =-=[15]-=- and Kikuchi [16]. For the important special case of the standard BP algorithm, we show that its fixed points are the same as the stationary points of the Bethe free energy, thus establishing an impor... |

41 |
The generalized distributive law and free energy minimization
- Aji, McEliece
- 2003
(Show Context)
Citation Context ...uced region-based free energies and GBP algorithms based on the cluster variation method, Aji and McEliece introduced a class of free energy approximations and GBP algorithms based on junction graphs =-=[27]-=-. One of the goals of this paper is to unify our previous approach with the one that Aji and McEliece presented. McEliece and Yildirim have independently developed a unified approach Here is an index ... |

37 |
Introduction to inference for Bayesian networks
- Cowell
(Show Context)
Citation Context ... of cycles. Thus, a common approach for dealing with graphical models that do have cycles is to try to convert them to equivalent cyclefree graphical models, and then to use the standard BP algorithm =-=[13]-=-. In some cases, this is possible, but for many other cases of practical interest, such an approach is intractable, and one must settle for approximate methods. Fortunately, the standard BP algorithms... |

31 |
Infinite-ranged models of Spin Glasses
- Kirkpatrick, Sherrington
- 1978
(Show Context)
Citation Context ...riable nodes, where every pair of nodes is connected by a factor. (A version of this factor graph with random factors is known in the physics literature as the Sherrington-Kirpatrick Ising spin glass =-=[54]-=-.) Now take, as the regions to include in the region graph, every triplet of nodes (and all three factors that connect them), every pair of nodes (and the factor that connects them), and every single ... |

30 | Fractional belief propagation
- Wiegerinck, Heskes
- 2002
(Show Context)
Citation Context ...sed in the Bethe approximation, but modifying the entropic counting numbers to differ from those given in the Bethe approximation, is one way of deriving the “fractional belief propagation algorithm” =-=[40]-=- and the essentially equivalent “convexified Bethe free energy” [41] approximation. In this paper, we will always assume just one set of counting numbers. In fact, not all region-based approximations ... |

25 |
Theory of the frustration effect in spin glasses
- Toulouse
- 1977
(Show Context)
Citation Context ... 2 and 3 prefers them to be in different states. Not all of these factors can be satisfied simultaneously; this is thus a very simple example of what statistical physicists call a “frustrated” sys=-=tem [42]. The be-=-liefs ba(xa) and bi(xi) that minimize the constrained Bethe free energy for this model are � � 0.4 0.1 bA(x1, x2) = bB(x1, x3) = , (50) 0.1 0.4 � � 0.1 0.4 bC(x1, x2) = , (51) 0.4 0.1 9 and b1... |

23 | Belief propagation on partially ordered sets
- McEliece, Yildirim
- 2002
(Show Context)
Citation Context ...beling functions , where the function has arguments that are some subset of . to belief propagation which is largely equivalent to our region graph approach, and we recommend their elegant exposition =-=[28]-=-. Pakzad and Anantharam have also recently presented parallel ideas in a brief paper [29]. The outline for the rest of the paper is as follows. In section II, we review and introduce our notation for ... |

23 | Belief propagation and statistical physics
- Pakzad, Anantharam
- 2002
(Show Context)
Citation Context ...ropagation which is largely equivalent to our region graph approach, and we recommend their elegant exposition [28]. Pakzad and Anantharam have also recently presented parallel ideas in a brief paper =-=[29]-=-. The outline for the rest of the paper is as follows. In section II, we review and introduce our notation for factor graphs and the standard BP algorithm. In sections III and IV, we introduce and exp... |

21 |
On the choice of regions for generalized belief propagation
- Welling
- 2004
(Show Context)
Citation Context ...and automatically construct maxent-normal region graph approximations that also satisfied our heuristics. We do not know of any such method, but we refer the reader to an interesting paper by Welling =-=[56], wh-=-o developed a “bottom-up” approach to generating region graph approximations starting from the Bethe approximation. IX. GENERALIZED BELIEF PROPAGATION ALGORITHMS We have already seen that the stat... |

19 | A novel iteration scheme for the cluster variation method
- Kappen, Wiegerinck
- 2002
(Show Context)
Citation Context ...s are in a feasible set. Based on the equivalence, first noted in our earlier work [17], others have recently devised algorithms that directly minimize the free energy on the feasible set [35], [36], =-=[37]-=-. Such free energy minimizations are somewhat slower than the BP algorithm, but they are guaranteed to converge. VI. THE REGION GRAPH METHOD We now introduce region graphs, which are central to the re... |