Results 1  10
of
21
Convergent message passing algorithms  a unifying view
 In Proc. Twentyeighth Conference on Uncertainty in Artificial Intelligence (UAI ’09
, 2009
"... Messagepassing algorithms have emerged as powerful techniques for approximate inference in graphical models. When these algorithms converge, they can be shown to find local (or sometimes even global) optima of variational formulations to the inference problem. But many of the most popular algorithm ..."
Abstract

Cited by 20 (0 self)
 Add to MetaCart
Messagepassing algorithms have emerged as powerful techniques for approximate inference in graphical models. When these algorithms converge, they can be shown to find local (or sometimes even global) optima of variational formulations to the inference problem. But many of the most popular algorithms are not guaranteed to converge. This has lead to recent interest in convergent messagepassing algorithms. In this paper, we present a unified view of convergent messagepassing algorithms. We algorithm, treeconsistency bound optimization (TCBO) that is provably convergent in both its sum and max product forms. We then show that many of the existing convergent algorithms are instances of our TCBO algorithm, and obtain novel convergent algorithms “for free ” by exchanging maximizations and summations in existing algorithms. In particular, we show that Wainwright’s nonconvergent sumproduct algorithm for tree based variational bounds, is actually convergent with the right update order for the case where trees are monotonic chains. 1
An Alternating Direction Method for Dual MAP LP Relaxation
"... Abstract. Maximum aposteriori (MAP) estimation is an important task in many applications of probabilistic graphical models. Although finding an exact solution is generally intractable, approximations based on linear programming (LP) relaxation often provide good approximate solutions. In this paper ..."
Abstract

Cited by 13 (1 self)
 Add to MetaCart
Abstract. Maximum aposteriori (MAP) estimation is an important task in many applications of probabilistic graphical models. Although finding an exact solution is generally intractable, approximations based on linear programming (LP) relaxation often provide good approximate solutions. In this paper we present an algorithm for solving the LP relaxation optimization problem. In order to overcome the lack of strict convexity, we apply an augmented Lagrangian method to the dual LP. The algorithm, based on the alternating direction method of multipliers (ADMM), is guaranteed to converge to the global optimum of the LP relaxation objective. Our experimental results show that this algorithm is competitive with other stateoftheart algorithms for approximate MAP estimation.
A primaldual messagepassing algorithm for approximated large scale structured prediction
 In Advances in Neural Information Processing Systems 23
, 2010
"... In this paper we propose an approximated structured prediction framework for large scale graphical models and derive messagepassing algorithms for learning their parameters efficiently. We first relate CRFs and structured SVMs and show that in CRFs a variant of the logpartition function, known as ..."
Abstract

Cited by 12 (7 self)
 Add to MetaCart
In this paper we propose an approximated structured prediction framework for large scale graphical models and derive messagepassing algorithms for learning their parameters efficiently. We first relate CRFs and structured SVMs and show that in CRFs a variant of the logpartition function, known as the softmax, smoothly approximates the hinge loss function of structured SVMs. We then propose an intuitive approximation for the structured prediction problem, using duality, based on a local entropy approximation and derive an efficient messagepassing algorithm that is guaranteed to converge. Unlike existing approaches, this allows us to learn efficiently graphical models with cycles and very large number of parameters. 1
Variational algorithms for marginal map
 In UAI
, 2011
"... Marginal MAP problems are notoriously difficult tasks for graphical models. We derive a general variational framework for solving marginal MAP problems, in which we apply analogues of the Bethe, treereweighted, and mean field approximations. We then derive a “mixed ” message passing algorithm and a ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
Marginal MAP problems are notoriously difficult tasks for graphical models. We derive a general variational framework for solving marginal MAP problems, in which we apply analogues of the Bethe, treereweighted, and mean field approximations. We then derive a “mixed ” message passing algorithm and a convergent alternative using CCCP to solve the BPtype approximations. Theoretically, we give conditions under which the decoded solution is a global or local optimum, and obtain novel upper bounds on solutions. Experimentally we demonstrate that our algorithms outperform related approaches. We also show that EM and variational EM comprise a special case of our framework. 1
The Bethe Permanent of a NonNegative Matrix
, 2010
"... It has recently been observed that the permanent of a nonnegative matrix, i.e., of a matrix containing only nonnegative real entries, can very well be approximated by solving a certain Bethe free energy minimization problem with the help of the sumproduct algorithm. We call the resulting approxima ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
It has recently been observed that the permanent of a nonnegative matrix, i.e., of a matrix containing only nonnegative real entries, can very well be approximated by solving a certain Bethe free energy minimization problem with the help of the sumproduct algorithm. We call the resulting approximation of the permanent the Bethe permanent. In this paper we give reasons why this approach to approximating the permanent works well. Namely, we show that the Bethe free energy is a convex function and that the sumproduct algorithm finds its minimum efficiently. We also show that the permanent is lower bounded by the Bethe permanent, and we list some empirical evidence that the permanent is upper bounded by some constant (that modestly grows with the matrix size) times the Bethe permanent. Part of these results are obtained by a combinatorial characterization of the Bethe permanent in terms of permanents of socalled lifted versions of the matrix under consideration. We conclude the paper with some conjectures about permanentbased pseudocodewords and permanentbased kernels, and we comment on possibilities to modify the Bethe permanent so that it approximates the permanent even better.
Convex MaxProduct Algorithms for Continuous MRFs with Applications to Protein Folding
"... This paper investigates convex belief propagation algorithms for Markov random fields (MRFs) with continuous variables. Our first contribution is a theorem generalizing properties of the discrete case to the continuous case. Our second contribution is an algorithm for computing the value of the Lagr ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
This paper investigates convex belief propagation algorithms for Markov random fields (MRFs) with continuous variables. Our first contribution is a theorem generalizing properties of the discrete case to the continuous case. Our second contribution is an algorithm for computing the value of the Lagrangian relaxation of the MRF in the continuous case based on associating the continuous variables with an everfiner interval grid. A third contribution is a particle method which uses convex maxproduct in resampling particles. This last algorithm is shown to be particularly effective for protein folding where it outperforms particle methods based on standard maxproduct resampling. 1.
Continuous markov random fields for robust stereo estimation
 In arXiv:1204.1393v1
"... Abstract. In this paper we present a novel slantedplane model which reasons jointly about occlusion boundaries as well as depth. We formulate the problem as one of inference in a hybrid MRF composed of both continuous (i.e., slanted 3D planes) and discrete (i.e., occlusion boundaries) random variab ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
Abstract. In this paper we present a novel slantedplane model which reasons jointly about occlusion boundaries as well as depth. We formulate the problem as one of inference in a hybrid MRF composed of both continuous (i.e., slanted 3D planes) and discrete (i.e., occlusion boundaries) random variables. This allows us to define potentials encoding the ownership of the pixels that compose the boundary between segments, as well as potentials encoding which junctions are physically possible. Our approach outperforms the stateoftheart on Middlebury high resolution imagery [1] as well as in the more challenging KITTI dataset [2], while being more efficient than existing slanted plane MRF methods, taking on average 2 minutes to perform inference on high resolution imagery. 1
Efficiently Searching for Frustrated Cycles in MAP Inference
"... Dual decomposition provides a tractable framework for designing algorithms for finding the most probable (MAP) configuration in graphical models. However, for many realworld inference problems, the typical decomposition has a large integrality gap, due to frustrated cycles. One way to tighten the r ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
Dual decomposition provides a tractable framework for designing algorithms for finding the most probable (MAP) configuration in graphical models. However, for many realworld inference problems, the typical decomposition has a large integrality gap, due to frustrated cycles. One way to tighten the relaxation is to introduce additional constraints that explicitly enforce cycle consistency. Earlier work showed that clusterpursuit algorithms, which iteratively introduce cycle and other higherorder consistency constraints, allows one to exactly solve many hard inference problems. However, these algorithms explicitly enumerate a candidate set of clusters, limiting them to triplets or other short cycles. We solve the search problem for cycle constraints, giving a nearly linear time algorithm for finding the most frustrated cycle of arbitrary length. We show how to use this search algorithm together with the dual decomposition framework and clusterpursuit. The new algorithm exactly solves MAP inference problems arising from relational classification and stereo vision. 1
LPQP for MAP: Putting LP Solvers to Better Use
"... MAP inference for general energy functions remains a challenging problem. While most efforts are channeled towards improving the linear programming (LP) based relaxation, this work is motivated by the quadratic programming (QP) relaxation. We propose a novel MAP relaxation that penalizes the Kullbac ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
MAP inference for general energy functions remains a challenging problem. While most efforts are channeled towards improving the linear programming (LP) based relaxation, this work is motivated by the quadratic programming (QP) relaxation. We propose a novel MAP relaxation that penalizes the KullbackLeibler divergence between the LP pairwise auxiliary variables, and QP equivalent terms given by the product of the unaries. We develop two efficient algorithms based on variants of this relaxation. The algorithms minimize the nonconvex objective using belief propagation and dual decomposition as building blocks. Experiments on synthetic and realworld data show that the solutions returned by our algorithms substantially improve over the LP relaxation. 1.