## Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations

### Cached

### Download Links

Citations: | 79 - 10 self |

### BibTeX

@MISC{Globerson_fixingmax-product:,

author = {Amir Globerson and Tommi Jaakkola},

title = {Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations},

year = {}

}

### OpenURL

### Abstract

We present a novel message passing algorithm for approximating the MAP problem in graphical models. The algorithm is similar in structure to max-product but unlike max-product it always converges, and can be proven to find the exact MAP solution in various settings. The algorithm is derived via block coordinate descent in a dual of the LP relaxation of MAP, but does not require any tunable parameters such as step size or tree weights. We also describe a generalization of the method to cluster based potentials. The new method is tested on synthetic and real-world problems, and compares favorably with previous approaches. Graphical models are an effective approach for modeling complex objects via local interactions. In such models, a distribution over a set of variables is assumed to factor according to cliques of a graph with potentials assigned to each clique. Finding the assignment with highest probability in these models is key to using them in practice, and is often referred to as the MAP (maximum aposteriori) assignment problem. In the general case the problem is NP hard, with complexity exponential in the tree-width of the underlying graph.

### Citations

7092 |
Probabilistic reasoning in intelligent systems: networks of plausible inference
- Pearl
- 1988
(Show Context)
Citation Context ... problems [13]. The key problem with generic LP solvers is that they do not use the graph structure explicitly and thus may be sub-optimal in terms of computational efficiency. The max-product method =-=[7]-=- is a message passing algorithm that is often used to approximate the MAP problem. In contrast to generic LP solvers, it makes direct use of the graph structure in constructing and passing messages, a... |

414 | Constructing free-energy approximations and generalized belief propagation algorithms
- Yedidia, Freeman, et al.
(Show Context)
Citation Context ... [−cF,cF] respectively, and we used values of cI and cF in the range range [0.1, 2.35] (with intervals of 0.25), resulting in 100 different models. The clusters for GEMPLP were the faces of the graph =-=[14]-=-. To see if NMPLP converges to the LP solution we also used an LP solver to solve the LP relaxation. We found that the the normalized difference between NMPLP and LP objective was at most 10 −3 (media... |

303 | Convergent tree-reweighted message passing for energy minimization
- Kolmogorov
(Show Context)
Citation Context ...the exact MAP. A recent result [1] showed that for maximum weight matching on bi-partite graphs max-product and LP also yield the exact MAP [1]. Finally, Tree-Reweighted max-product (TRMP) algorithms =-=[5, 10]-=- were shown to converge to the LP solution for binary xi variables, as shown in [6]. In this work, we propose the Max Product Linear Programming algorithm (MPLP) - a very simple variation on max-produ... |

134 | MAP estimation via agreement on trees: message-passing and linear programming
- Wainwright, Jaakkola, et al.
- 2005
(Show Context)
Citation Context ...the exact MAP. A recent result [1] showed that for maximum weight matching on bi-partite graphs max-product and LP also yield the exact MAP [1]. Finally, Tree-Reweighted max-product (TRMP) algorithms =-=[5, 10]-=- were shown to converge to the LP solution for binary xi variables, as shown in [6]. In this work, we propose the Max Product Linear Programming algorithm (MPLP) - a very simple variation on max-produ... |

57 | Linear programming relaxations and belief propagation – an empirical study
- Yanover, Meltzer, et al.
(Show Context)
Citation Context ...the minimum cut problem and maximum weight matching in bi-partite graphs [8]. Although LP relaxations can be solved using standard LP solvers, this may be computationally intensive for large problems =-=[13]-=-. The key problem with generic LP solvers is that they do not use the graph structure explicitly and thus may be sub-optimal in terms of computational efficiency. The max-product method [7] is a messa... |

51 | On optimality of tree-reweighted max-product message-passing
- Kolmogorov, Wainwright
- 2005
(Show Context)
Citation Context ...e graphs max-product and LP also yield the exact MAP [1]. Finally, Tree-Reweighted max-product (TRMP) algorithms [5, 10] were shown to converge to the LP solution for binary xi variables, as shown in =-=[6]-=-. In this work, we propose the Max Product Linear Programming algorithm (MPLP) - a very simple variation on max-product that is guaranteed to converge, and has several advantageous properties. MPLP is... |

46 | Maximum Weight Matching via Max-Product Belief Propagation
- Bayati, Shah, et al.
- 2005
(Show Context)
Citation Context ...en max-product and the LP relaxation has remained largely elusive, although there are some notable exceptions: For tree-structured graphs, max-product and LP both yield the exact MAP. A recent result =-=[1]-=- showed that for maximum weight matching on bi-partite graphs max-product and LP also yield the exact MAP [1]. Finally, Tree-Reweighted max-product (TRMP) algorithms [5, 10] were shown to converge to ... |

45 | Structured prediction, dual extragradient and Bregman projections
- Taskar, Lacoste-Julien, et al.
- 2006
(Show Context)
Citation Context ...egral, it is guaranteed to be the exact MAP. For some classes of problems the LP relaxation is provably correct. These include the minimum cut problem and maximum weight matching in bi-partite graphs =-=[8]-=-. Although LP relaxations can be solved using standard LP solvers, this may be computationally intensive for large problems [13]. The key problem with generic LP solvers is that they do not use the gr... |

45 | T.: MAP estimation, linear programming and belief propagation with convex free energies
- Weiss, Yanover, et al.
- 2007
(Show Context)
Citation Context ...⊂ {1, . . .,n} (the set of clusters is denoted by C), and a function f(x; θ) = � c θc(xc) defined via potentials over clusters θc(xc). The MAP problem in this case also has an LP relaxation (see e.g. =-=[11]-=-). To define the LP we introduce the following definitions: S = {c∩ĉ : c, ĉ ∈ C, c∩ĉ �= ∅} is the set of intersection between clusters and S(c) = {s ∈ S : s ⊆ c} is the set of overlap sets for cluster... |

22 | Lagrangian relaxation for MAP estimation in graphical models
- Johnson, Malioutov, et al.
- 2007
(Show Context)
Citation Context ...nce to the dual optimum since coordinate descent algorithms may get stuck at a point that is not a global optimum. There are ways of overcoming this difficulty, for example by smoothing the objective =-=[4]-=- or using techniques as in [2] (see p. 636). We leave such extensions for further work. In this section we provide several results about the properties of the MPLP fixed points and their relation to t... |

18 | Towards low-complexity linearprogramming decoding
- Vontobel, Koetter
- 2006
(Show Context)
Citation Context ... presented a method for overcoming the local optimum problem, by smoothing the objective so that it is strictly convex. Such an approach could also be used within our algorithms. Vontobel and Koetter =-=[9]-=- recently introduced a coordinate descent algorithm for decoding LDPC codes. Their method is specifically tailored for this case, and uses updates that are similar to our edge based updates. Finally, ... |

12 | Convergent propagation algorithms via oriented trees
- Globerson, Jaakkola
- 2007
(Show Context)
Citation Context ...is specifically tailored for this case, and uses updates that are similar to our edge based updates. Finally, the concept of dual coordinate descent may be used in approximating marginals as well. In =-=[3]-=- we use such an approach to optimize a variational bound on the partition function. The derivation uses some of the ideas used in the MPLP dual, but importantly does not find the minimum for each coor... |

2 |
A linear programming approach to max-sum, a review
- Werner
- 2007
(Show Context)
Citation Context ...ther is its simple generalization to potentials over clusters of nodes (Sec. 6). Recently, several new dual LP algorithms have been introduced, which are more closely related to our formalism. Werner =-=[12]-=- presented a class of algorithms which also improve the dual LP at every iteration. The simplest of those is the max-sum-diffusion algorithm, which is similar to our EMPLP algorithm, although the upda... |

1 | xj) + λ −j i (xi) + λ −i j (xj) (8) � (9) � and a similar expression for βji - θij |