## Convergent Decomposition Solvers for Tree-reweighted Free Energies

Citations: | 1 - 0 self |

### BibTeX

@MISC{Jancsary_convergentdecomposition,

author = {Jeremy Jancsary and Gerald Matz},

title = {Convergent Decomposition Solvers for Tree-reweighted Free Energies},

year = {}

}

### OpenURL

### Abstract

We investigate minimization of treereweighted free energies for the purpose of obtaining approximate marginal probabilities and upper bounds on the partition function of cyclic graphical models. The solvers we present for this problem work by directly tightening tree-reweighted upper bounds. As a result, they are particularly efficient for tree-reweighted energies arising from a small number of spanning trees. While this assumption may seem restrictive at first, we show how small sets of trees can be constructed in a principled manner. An appealing property of our algorithms, which results from the problem decomposition, is that they are embarrassingly parallel. In contrast to the original message passing algorithm introduced for this problem, we obtain global convergence guarantees. 1

### Citations

740 | Nonlinear Programming. Athena Scientific - Bertsekas - 1999 |

298 | Understanding belief propagation and its generalizations - Yedidia, Freeman, et al. - 2003 |

215 |
Updating quasi-Newton matrices with limited storage
- Nocedal
- 1980
(Show Context)
Citation Context ...d by Wang et al. (2005). 3.4.2 Projected Quasi-Newton Method The projected quasi-Newton (PQN) method was recently introduced by Schmidt et al. (2009) and can be considered a generalization of L-BFGS (=-=Nocedal, 1980-=-) to constrained optimization. At each iteration, a feasible direction is found by minimizing a quadratic model subject to the original constraints: min ⃗θ∈C(θ) ˜Φ (k) +( ⃗ θ− ⃗ θ (k) )·∇ (k) 1 ⃗θ + 2... |

161 |
Two point step size gradient method
- Barzilai, Borwein
- 1988
(Show Context)
Citation Context ...Projected Gradient Method The main improvements of the spectral projected gradient (SPG) method (Birgin et al., 2000) over classic projected gradient descent are a particular choice of the step size (=-=Barzilai and Borwein, 1988-=-) and a nonmonotone, yet convergent line search (Grippo et al., 1986). In the setting of unconstrained quadratics, the SPG algorithm has been observed to converge superlinearly towards the optimum. We... |

154 | A new class of upper bounds on the log partition function - Wainwright, Jaakkola, et al. - 2005 |

148 | A Nonmonotone Line Search Technique for Newton's Method - Grippo, Lampariello, et al. - 1986 |

135 | Nonmonotone Spectral Projected Gradient Methods on Convex Sets - Birgin, Martínez, et al. |

108 | MAP Estimation via Agreement on (Hyper)Trees: Message-Passing and Linear-Programming Approaches - Wainwright, Jaakkola, et al. - 2005 |

103 | Representations of quasi-Newton matrices and their use in limited memory methods
- Byrd, Nocedal, et al.
- 1994
(Show Context)
Citation Context ...⃗θ + 2 (⃗ θ− ⃗ θ (k) ) T B (k) ( ⃗ θ− ⃗ θ (k) ) where B (k) is a positive-definitive approximation to the Hessian that is maintained in compact form in terms of the previous p iterates and gradients (=-=Byrd et al., 1994-=-). The SPG algorithm can be used to perform the above minimization effectively. We hypothesized that PQN might compensate for the larger per-iteration cost through improved asymptotic convergence and ... |

74 | MRF optimization via dual decomposition: Messagepassing revisited - Komodakis, Paragios, et al. - 2007 |

32 | libDAI: A free and open source C++ library for discrete approximate inference in graphical models - Mooij - 2010 |

26 | Convexity arguments for efficient minimization of the Bethe and Kikuchi free energies - Heskes - 2006 |

24 | Optimizing costly functions with simple constraints: A limited-memory projected quasi-newton algorithm - Schmidt, Berg, et al. - 2009 |

21 |
Accelerated dual decomposition for MAP inference
- Jojic, Gould, et al.
- 2010
(Show Context)
Citation Context ...pper bound, hence primal solutions must be generated heuristically. Moreover, the upper bound on the MAP score is non-differentiable, which has recently been dealt with using proximal regularization (=-=Jojic et al., 2010-=-). On the other hand, the upper bound on the log partition function depends on the choice of trees, a different source of complication. Several independent lines of work have focused on convergent alg... |

20 | Convergent message passing algorithms - a unifying view - Meltzer, Globerson, et al. - 2009 |

15 | Convergent messagepassing algorithms for inference over general graphs with convex free energies - Hazan, Shashua - 2008 |

13 | Complexity of inference in graphical models
- Chandrasekaran, Srebro, et al.
- 2008
(Show Context)
Citation Context ...ntees. 1 INTRODUCTION Exact computation of marginal probabilities and the partition function in general graphical models is an NP-hard problem that scales exponentially in the treewidth of the graph (=-=Chandrasekaran et al., 2008-=-). Much effort has been put into construction of approximate inference algorithms that remain tractable even for graphs of large treewidth, such as those involving many cycles. Good results were initi... |

12 | Convergent propagation algorithms via oriented trees - Globerson, Jaakkola - 2007 |

2 | Exploring artificial intelligence in the new millennium - In - 2003 |

1 | Approximately optimal utility maximization - Nedić, Subramanian - 2009 |

1 | TREE SELECTION We start with a general discussion, as Proposition 2 and Proposition 3 are both based on the same framework. In particular, both algorithms seek the solution to a convex optimization problem min ν∈T(G) f(ν), where f(·) is a convex function - T - 1971 |