• Documents
  • Authors
  • Tables
  • Log in
  • Sign up
  • MetaCart
  • DMCA
  • Donate

CiteSeerX logo

Advanced Search Include Citations
Advanced Search Include Citations

CCCP algorithms to minimize the Bethe and Kikuchi free energies: Convergent alternatives to belief propagation. (2002)

by Raymond, Ricci-Tersenghi A L Yuille
Venue:Neural Computation,
Add To MetaCart

Tools

Sorted by:
Results 1 - 10 of 137
Next 10 →

Information Theory, Inference, and Learning Algorithms

by David J. C. MacKay , 2003
"... ..."
Abstract - Cited by 1936 (13 self) - Add to MetaCart
Abstract not found

Graphical models, exponential families, and variational inference

by Martin J. Wainwright, Michael I. Jordan , 2008
"... The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical models have become a focus of research in many statistical, computational and mathematical fiel ..."
Abstract - Cited by 819 (28 self) - Add to MetaCart
The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical models have become a focus of research in many statistical, computational and mathematical fields, including bioinformatics, communication theory, statistical physics, combinatorial optimization, signal and image processing, information retrieval and statistical machine learning. Many problems that arise in specific instances — including the key problems of computing marginals and modes of probability distributions — are best studied in the general setting. Working with exponential family representations, and exploiting the conjugate duality between the cumulant function and the entropy for exponential families, we develop general variational representations of the problems of computing likelihoods, marginal probabilities and most probable configurations. We describe how a wide varietyof algorithms — among them sum-product, cluster variational methods, expectation-propagation, mean field methods, max-product and linear programming relaxation, as well as conic programming relaxations — can all be understood in terms of exact or approximate forms of these variational representations. The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.
(Show Context)

Citation Context

...entials, Dobrushin-type conditions and related contraction arguments guarantee both convergence of the updates, and as a consequence, uniqueness of the associated fixed point. A parallel line of work =-=[111, 254, 270]-=- has explored alternatives to sum-product that are guaranteed to converge, albeit at the price of increased computational cost. However, with the exception of trees and other special cases [110, 167, ...

Constructing Free Energy Approximations and Generalized Belief Propagation Algorithms

by Jonathan S. Yedidia, William T. Freeman, Yair Weiss - IEEE Transactions on Information Theory , 2005
"... Important inference problems in statistical physics, computer vision, error-correcting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems t ..."
Abstract - Cited by 585 (13 self) - Add to MetaCart
Important inference problems in statistical physics, computer vision, error-correcting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree, but only approximate when the factor graph has cycles. We show that BP fixed points correspond to the stationary points of the Bethe approximation of the free energy for a factor graph. We explain how to obtain regionbased free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms. We emphasize the conditions a free energy approximation must satisfy in order to be a “valid ” or “maxent-normal ” approximation. We describe the relationship between four different methods that can be used to generate valid approximations: the “Bethe method, ” the “junction graph method, ” the “cluster variation method, ” and the “region graph method.” Finally, we explain how to tell whether a region-based approximation, and its corresponding GBP algorithm, is likely to be accurate, and describe empirical results showing that GBP can significantly outperform BP.
(Show Context)

Citation Context

...t the beliefs are in a feasible set. Based on the equivalence, first noted in our earlier work [17], others have recently devised algorithms that directly minimize the free energy on the feasible set =-=[35]-=-, [36], [37]. Such free energy minimizations are somewhat slower than the BP algorithm, but they are guaranteed to converge. VI. THE REGION GRAPH METHOD We now introduce region graphs, which are centr...

Computer Vision: Algorithms and Applications

by Richard Szeliski , 2010
"... ..."
Abstract - Cited by 252 (2 self) - Add to MetaCart
Abstract not found

Collective classification in network data

by Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, Tina Eliassi-Rad , 2008
"... Numerous real-world applications produce networked data such as web data (hypertext documents connected via hyperlinks) and communication networks (people connected via communication links). A recent focus in machine learning research has been to extend traditional machine learning classification te ..."
Abstract - Cited by 178 (32 self) - Add to MetaCart
Numerous real-world applications produce networked data such as web data (hypertext documents connected via hyperlinks) and communication networks (people connected via communication links). A recent focus in machine learning research has been to extend traditional machine learning classification techniques to classify nodes in such data. In this report, we attempt to provide a brief introduction to this area of research and how it has progressed during the past decade. We introduce four of the most widely used inference algorithms for classifying networked data and empirically compare them on both synthetic and real-world data.
(Show Context)

Citation Context

...les of approximate inference algorithms include algorithms developed to extend and improve loopy belief propagation to remove some of its shortcomings such as alternatives with convergence guarantees =-=[67]-=- and alternatives that go beyond just using edge and node marginals to compute more accurate marginal probability estimates such as the cluster variational method [66], junction graph method [1] and r...

Tree-Based Reparameterization Framework for Analysis of Belief Propagation and Related Algorithms

by Martin Wainwright, Tommi Jaakkola, Alan Willsky , 2001
"... We present a tree-based reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sum-product algorithm [39, 36], as well as a rich set of variations and ext ..."
Abstract - Cited by 122 (20 self) - Add to MetaCart
We present a tree-based reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sum-product algorithm [39, 36], as well as a rich set of variations and extensions of belief propagation. Algorithms in this class can be formulated as a sequence of reparameterization updates, each of which entails re-factorizing a portion of the distribution corresponding to an acyclic subgraph (i.e., a tree). The ultimate goal is to obtain an alternative but equivalent factorization using functions that represent (exact or approximate) marginal distributions on cliques of the graph. Our framework highlights an important property of BP and the entire class of reparameterization algorithms: the distribution on the full graph is not changed. The perspective of tree-based updates gives rise to a simple and intuitive characterization of the fixed points in terms of tree consistency. We develop interpretations of these results in terms of information geometry. The invariance of the distribution, in conjunction with the fixed point characterization, enables us to derive an exact relation between the exact marginals on an arbitrary graph with cycles, and the approximations provided by belief propagation, and more broadly, any algorithm that minimizes the Bethe free energy. We also develop bounds on this approximation error, which illuminate the conditions that govern their accuracy. Finally, we show how the reparameterization perspective extends naturally to more structured approximations (e.g., Kikuchi and variants [52, 37]) that operate over higher order cliques.
(Show Context)

Citation Context

...rgy associated with the graphicalsWAINWRIGHT et al.: ANALYSIS OF SUM-PRODUCT AND RELATED ALGORITHMS BY TREE-BASED REPARAMETERIZATION 1121 distribution, 1 which inspired other researchers (e.g., [22], =-=[23]-=-) to develop more sophisticated algorithms for minimizing the Bethe free energy. Yedidia et al. also proposed extensions to BP based on cluster variational methods [24]; in subsequent work, various re...

Finding deformable shapes using loopy belief propogation.

by J Coughlan, S J Ferreira - In European Conference on Computer Vision, , 2002
"... ..."
Abstract - Cited by 107 (1 self) - Add to MetaCart
Abstract not found
(Show Context)

Citation Context

... MAP marginal estimate will approach the MAP estimate, and we speculate that this is the case for our application.) We can estimate the MAP marginal using the BP (or alternate algorithms such as CCCP =-=[21]-=-). (An alternate version of BP, the max-product belief propagation algorithm, may also be used to estimate the MAP directly.) We note that the MAP marginal estimate may not suffice for multiple target...

Loopy Belief Propagation and Gibbs Measures

by Sekhar C. Tatikonda, Michael I. Jordan - In Uncertainty in Artificial Intelligence , 2002
"... We address the question of convergence in the loopy belief propagation (LBP) algorithm. ..."
Abstract - Cited by 103 (6 self) - Add to MetaCart
We address the question of convergence in the loopy belief propagation (LBP) algorithm.
(Show Context)

Citation Context

...any new methods that generalize the basic algorithm have been developed. Some of these methods include Kikuchi based methods [6], the tree reparameterization technique [4], and the double-loop scheme =-=[7]-=-. Also it has been shown that the LBP algorithm can be viewed an iterative descent down an associated Bethe free energy. [6] These advances notwithstanding, a complete understanding of the convergence...

Recovering Occlusion Boundaries from a Single Image

by Derek Hoiem, Andrew N. Stein, Alexei A. Efros, Martial Hebert
"... Occlusion reasoning, necessary for tasks such as navigation and object search, is an important aspect of everyday life and a fundamental problem in computer vision. We believe that the amazing ability of humans to reason about occlusions from one image is based on an intrinsically 3D interpretation. ..."
Abstract - Cited by 93 (10 self) - Add to MetaCart
Occlusion reasoning, necessary for tasks such as navigation and object search, is an important aspect of everyday life and a fundamental problem in computer vision. We believe that the amazing ability of humans to reason about occlusions from one image is based on an intrinsically 3D interpretation. In this paper, our goal is to recover the occlusion boundaries and depth ordering of free-standing structures in the scene. Our approach is to learn to identify and label occlusion boundaries using the traditional edge and region cues together with 3D surface and depth cues. Since some of these cues require good spatial support (i.e., a segmentation), we gradually create larger regions and use them to improve inference over the boundaries. Our experiments demonstrate the power of a scene-based approach to occlusion reasoning. 1.
(Show Context)

Citation Context

... obtain “soft-max” likelihood estimates. To do this, we combine the sum-product algorithm of Heskes et al. [9] (based on the Kikuchi free energy) with the mean field approximation suggested by Yuille =-=[31]-=- (raise , with T =0.5in our experiments). each factor to the 1 T 4. Segmentation from Boundary Likelihoods Given a soft boundary map, we can compute a hierarchical segmentation and threshold it to get...

On the Uniqueness of Loopy Belief Propagation Fixed Points

by Tom Heskes , 2004
"... We derive sufficient conditions for the uniqueness of loopy belief propagation fixed points. These conditions depend on both the structure of the graph and the strength of the potentials and naturally extend those for convexity of the Bethe free energy. We compare them with (a strengthened version o ..."
Abstract - Cited by 79 (2 self) - Add to MetaCart
We derive sufficient conditions for the uniqueness of loopy belief propagation fixed points. These conditions depend on both the structure of the graph and the strength of the potentials and naturally extend those for convexity of the Bethe free energy. We compare them with (a strengthened version of) conditions derived elsewhere for pairwise potentials. We discuss possible implications for convergent algorithms, as well as for other approximate free energies.
Powered by: Apache Solr
  • About CiteSeerX
  • Submit and Index Documents
  • Privacy Policy
  • Help
  • Data
  • Source
  • Contact Us

Developed at and hosted by The College of Information Sciences and Technology

© 2007-2019 The Pennsylvania State University