Results 1 
8 of
8
Convergent messagepassing algorithms for inference over general graphs with convex free energy
 In The 24th Conference on Uncertainty in Artificial Intelligence (UAI
, 2008
"... Inference problems in graphical models can be represented as a constrained optimization of a free energy function. It is known that when the Bethe free energy is used, the fixedpoints of the belief propagation (BP) algorithm correspond to the local minima of the free energy. However BP fails to conv ..."
Abstract

Cited by 15 (5 self)
 Add to MetaCart
Inference problems in graphical models can be represented as a constrained optimization of a free energy function. It is known that when the Bethe free energy is used, the fixedpoints of the belief propagation (BP) algorithm correspond to the local minima of the free energy. However BP fails to converge in many cases of interest. Moreover, the Bethe free energy is nonconvex for graphical models with cycles thus introducing great difficulty in deriving efficient algorithms for finding local minima of the free energy for general graphs. In this paper we introduce two efficient BPlike algorithms, one sequential and the other parallel, that are guaranteed to converge to the global minimum, for any graph, over the class of energies known as ”convex free energies”. In addition, we propose an efficient heuristic for setting the parameters of the convex free energy based on the structure of the graph. 1
Convergent propagation algorithms via oriented trees
 In UAI. 2007
"... Inference problems in graphical models are often approximated by casting them as constrained optimization problems. Message passing algorithms, such as belief propagation, have previously been suggested as methods for solving these optimization problems. However, there are few convergence guarantees ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
Inference problems in graphical models are often approximated by casting them as constrained optimization problems. Message passing algorithms, such as belief propagation, have previously been suggested as methods for solving these optimization problems. However, there are few convergence guarantees for such algorithms, and the algorithms are therefore not guaranteed to solve the corresponding optimization problem. Here we present an oriented tree decomposition algorithm that is guaranteed to converge to the global optimum of the TreeReweighted (TRW) variational problem. Our algorithm performs local updates in the convex dual of the TRW problem – an unconstrained generalized geometric program. Primal updates, also local, correspond to oriented reparametrization operations that leave the distribution intact. 1
Learning and evaluating Boltzmann machines
, 2008
"... We provide a brief overview of the variational framework for obtaining deterministic approximations or upper bounds for the logpartition function. We also review some of the Monte Carlo based methods for estimating partition functions of arbitrary Markov Random Fields. We then develop an annealed i ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
We provide a brief overview of the variational framework for obtaining deterministic approximations or upper bounds for the logpartition function. We also review some of the Monte Carlo based methods for estimating partition functions of arbitrary Markov Random Fields. We then develop an annealed importance sampling (AIS) procedure for estimating partition functions of restricted Boltzmann machines (RBM’s), semirestricted Boltzmann machines (SRBM’s), and Boltzmann machines (BM’s). Our empirical results indicate that the AIS procedure provides much better estimates of the partition function than some of the popular variationalbased methods. Finally, we develop a new learning algorithm for training general Boltzmann machines and show that it can be successfully applied to learning good generative models. Learning and Evaluating Boltzmann Machines
Convexifying the bethe free energy
 in Conference on Uncertainty in Artifical Intelligence (UAI
, 2009
"... The introduction of loopy belief propagation (LBP) revitalized the application of graphical models in many domains. Many recent works present improvements on the basic LBP algorithm in an attempt to overcome convergence and local optima problems. Notable among these are convexified free energy appro ..."
Abstract

Cited by 9 (2 self)
 Add to MetaCart
The introduction of loopy belief propagation (LBP) revitalized the application of graphical models in many domains. Many recent works present improvements on the basic LBP algorithm in an attempt to overcome convergence and local optima problems. Notable among these are convexified free energy approximations that lead to inference procedures with provable convergence and quality properties. However, empirically LBP still outperforms most of its convex variants in a variety of settings, as we also demonstrate here. Motivated by this fact we seek convexified free energies that directly approximate the Bethe free energy. We show that the proposed approximations compare favorably with stateofthe art convex free energy approximations. 1
Bounding the partition function using hölders inequality
, 2011
"... We describe an algorithm for approximate inference in graphical models based on Hölder’s inequality that provides upper and lower bounds on common summation problems such as computing the partition function or probability of evidence in a graphical model. Our algorithm unifies and extends several ex ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
We describe an algorithm for approximate inference in graphical models based on Hölder’s inequality that provides upper and lower bounds on common summation problems such as computing the partition function or probability of evidence in a graphical model. Our algorithm unifies and extends several existing approaches, including variable elimination techniques such as minibucket elimination and variational methods such as tree reweighted belief propagation and conditional entropy decomposition. We show that our method inherits benefits from each approach to provide significantly better bounds on sumproduct tasks. 1.
Variational Upper and Lower Bounds for Probabilistic Graphical Models
"... Probabilistic phylogenetic models which relax the site independence evolution assumption often face the problem of infeasible likelihood computations, for example, for the task of selecting suitable parameters for the model. We present a new approximation method, applicable for a wide range of proba ..."
Abstract
 Add to MetaCart
Probabilistic phylogenetic models which relax the site independence evolution assumption often face the problem of infeasible likelihood computations, for example, for the task of selecting suitable parameters for the model. We present a new approximation method, applicable for a wide range of probabilistic models, which guarantees to upper and lower bound the true likelihood of data, and apply it to the problem of probabilistic phylogenetic models. The new method is complementary to known variational methods that lower bound the likelihood, and it uses similar methods to optimize the bounds from above and below. We applied our method to aligned DNA sequences of various lengths from human in the region of the CFTR gene and homologous from eight mammals, and found the bounds to be appreciably close to the true likelihood whenever it could be computed. When computing the exact likelihood was not feasible, we demonstrated the proximity of the upper and lower variational bounds, implying a tight approximation of the likelihood. Key words: algorithms, computational molecular biology, genetic mapping, learning, secondary structure.
Dual Decomposition for Marginal Inference
"... We present a dual decomposition approach to the treereweighted belief propagation objective. Each tree in the treereweighted bound yields one subproblem, which can be solved with the sumproduct algorithm. The master problem is a simple differentiable optimization, to which a standard optimization ..."
Abstract
 Add to MetaCart
We present a dual decomposition approach to the treereweighted belief propagation objective. Each tree in the treereweighted bound yields one subproblem, which can be solved with the sumproduct algorithm. The master problem is a simple differentiable optimization, to which a standard optimization method can be applied. Experimental results on 10x10 Ising models show the dual decomposition approach using LBFGS is similar in settings where messagepassing converges quickly, and one to two orders of magnitude faster in settings where messagepassing requires many iterations, specifically high accuracy convergence, and strong interactions.
Contents lists available at SciVerse ScienceDirect Information Processing Letters
"... www.elsevier.com/locate/ipl Recursive sum–product algorithm for generalized outerplanar graphs ..."
Abstract
 Add to MetaCart
www.elsevier.com/locate/ipl Recursive sum–product algorithm for generalized outerplanar graphs