## Using combinatorial optimization within max-product belief propagation (2007)

### Cached

### Download Links

Venue: | Advances in Neural Information Processing Systems (NIPS |

Citations: | 39 - 6 self |

### BibTeX

@INPROCEEDINGS{Duchi07usingcombinatorial,

author = {John Duchi and Daniel Tarlow and Gal Elidan and Daphne Koller},

title = {Using combinatorial optimization within max-product belief propagation},

booktitle = {Advances in Neural Information Processing Systems (NIPS},

year = {2007}

}

### Years of Citing Articles

### OpenURL

### Abstract

In general, the problem of computing a maximum a posteriori (MAP) assignment in a Markov random field (MRF) is computationally intractable. However, in certain subclasses of MRF, an optimal or close-to-optimal assignment can be found very efficiently using combinatorial optimization algorithms: certain MRFs with mutual exclusion constraints can be solved using bipartite matching, and MRFs with regular potentials can be solved using minimum cut methods. However, these solutions do not apply to the many MRFs that contain such tractable components as sub-networks, but also other non-complying potentials. In this paper, we present a new method, called COMPOSE, for exploiting combinatorial optimization for sub-networks within the context of a max-product belief propagation algorithm. COMPOSE uses combinatorial optimization for computing exact maxmarginals for an entire sub-network; these can then be used for inference in the context of the network as a whole. We describe highly efficient methods for computing max-marginals for subnetworks corresponding both to bipartite matchings and to regular networks. We present results on both synthetic and real networks encoding correspondence problems between images, which involve both matching constraints and pairwise geometric constraints. We compare to a range of current methods, showing that the ability of COMPOSE to transmit information globally across the network leads to improved convergence, decreased running time, and higher-scoring assignments. 1

### Citations

7347 |
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
- Pearl
- 1988
(Show Context)
Citation Context ...bility of COMPOSE to transmit information globally across the network leads to improved convergence, decreased running time, and higher-scoring assignments. 1 Introduction Markov random fields (MRFs) =-=[12]-=- have been applied to a wide variety of real-world problems. However, the probabilistic inference task in MRFs — computing the posterior distribution of one or more variables — is tractable only in sm... |

2736 | Normalized cuts and image segmentation
- Shi, Malik
- 2000
(Show Context)
Citation Context ...g algorithms. These types of networks have been shown to be applicable in a variety of applications, such as stereo reconstruction [13] and segmentation for regular networks, and image correspondence =-=[15]-=- or word alignment for matching networks [19].sIn many real-world applications, however, the problem formulation does not fall neatly into one of these tractable subclasses. The problem may well have ... |

1468 | Efcient approximate energy minimization via graph cuts
- Boykov, Veksler, et al.
- 2001
(Show Context)
Citation Context ..., including to the class of networks with non-binary variables whose negative-log-probability is a convex function [5]. Moreover, for a range of conditions on the potentials, an α-expansion procedure =-=[2]-=-, which iteratively applies a mincut to a series of graphs, can be used to find a solution with guaranteed approximation error relative to the optimal MAP assignment. As above, a single joint assignme... |

739 | What energy functions can be minimized via graph cuts
- Kolmogorov, Zabih
- 2002
(Show Context)
Citation Context ...· φij(Xi = 0, Xj = 0) ≥ φij(Xi = 0, Xj = 1) · φij(Xi = 1, Xj = 0). For MRFs with only regular potentials, the MAP solution can be found as the minimum cut of a weighted graph constructed from the MRF =-=[9]-=-. This construction can be extended in various ways (see [9] for a survey), including to the class of networks with non-binary variables whose negative-log-probability is a convex function [5]. Moreov... |

492 | Loopy belief propagation for approximate inference: an empirical study
- Murphy, Weiss, et al.
- 1999
(Show Context)
Citation Context ...h are not often an appropriate model in practice. Thus, one typically must resort to the use of approximate inference methods, most commonly (in recent years) some variant of loopy belief propagation =-=[11]-=-. An alternative approach, whose popularity has grown in recent years, is based on the maximum a posteriori (MAP) inference problem — computing the single most likely assignment relative to the distri... |

262 | A comparative study of energy minimization methods for markov random fields with smoothness-based priors
- Szeliski, Zabih, et al.
(Show Context)
Citation Context ... MRF structures associated with the tractable components are quite dense and contain many small loops, leading to convergence problems and bad approximations. Indeed, recent empirical studies studies =-=[17]-=- show that belief propagation methods perform considerably worse than min-cut-based methods when applied to a variety of (purely) regular MRFs. Thus, falling back on belief propagation methods for the... |

189 | W.T.: On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs
- Weiss, Freeman
- 2001
(Show Context)
Citation Context ...n be shown to be intractable; we describe two such important classes in Section 4.sIn general, however, an exact solution to the MAP problem is also intractable. Max-product belief propagation (MPBP) =-=[20]-=- is a commonly-used method for finding an approximate solution. In this algorithm, each node Xi passes to its neighboring nodes Ni a message which is a vector defining a value for each value xi: ⎡ ⎤ δ... |

182 |
Algorithm Design
- Kleinberg, Tardos
- 2005
(Show Context)
Citation Context ...tching problems. Nevertheless, finding the maximum score bipartite matching (with any set of degree constraints) can be accomplished easily using standard combinatorial optimization algorithms (e.g., =-=[6]-=-). However, we also need to find all the max-marginals. Fortunately, we can adapt the standard algorithm for finding a single best matching to also find all of the max-marginals. A standard solution t... |

176 | C.: Learning structured prediction models: a large margin approach
- Taskar, Chatalbashev, et al.
- 2005
(Show Context)
Citation Context ...en shown to be applicable in a variety of applications, such as stereo reconstruction [13] and segmentation for regular networks, and image correspondence [15] or word alignment for matching networks =-=[19]-=-.sIn many real-world applications, however, the problem formulation does not fall neatly into one of these tractable subclasses. The problem may well have a large component that can be well-modeled as... |

159 | Exact optimization for Markov random fields with convex priors
- Ishikawa
- 2003
(Show Context)
Citation Context ...the MRF [9]. This construction can be extended in various ways (see [9] for a survey), including to the class of networks with non-binary variables whose negative-log-probability is a convex function =-=[5]-=-. Moreover, for a range of conditions on the potentials, an α-expansion procedure [2], which iteratively applies a mincut to a series of graphs, can be used to find a solution with guaranteed approxim... |

100 |
The quadratic assignment problem
- Lawler
- 1963
(Show Context)
Citation Context ...would be interesting to compare COMPOSE and these methods on a range of networks containing regular subgraphs. Our work is also related to work trying to solve the quadratic assignment problem (QAP) =-=[10]-=-, a class of problems of which our generalized matching networks are a special case. Standard algorithms for QAP include simulated annealing, tabu search, branch and bound, and ant algorithms [16]; th... |

78 | The correlated correspondence algorithm for unsupervised registration of nonrigid surfaces
- Anguelov, Srinivasan, et al.
- 2004
(Show Context)
Citation Context ...registering features between two images or 3D scans, we may formulate the task as a matching problem, but may also want to encode constraints that enforce the preservation of local or global geometry =-=[1]-=-. Unfortunately, once the network contains some “non-complying” potentials, it is not clear if and how one can apply the combinatorial optimization algorithm, even if only as a subroutine. In practice... |

78 | Learning associative markov networks
- Taskar, Chatalbashev, et al.
- 2004
(Show Context)
Citation Context ...using combinatorial optimization algorithms, even though posterior probability inference is intractable. So far, two main such classes of networks have been studied. Regular (or associative) networks =-=[18]-=-, where the potentials encode a preference for adjacent variables to take the same value, can be solved optimally or almost optimally using a minimum cut algorithm. Conversely, matching networks, wher... |

57 | ACO algorithms for the quadratic assignment problem. New ideas in optimization, pp
- Stutzle, Dorigo
- 1999
(Show Context)
Citation Context ...AP) [10], a class of problems of which our generalized matching networks are a special case. Standard algorithms for QAP include simulated annealing, tabu search, branch and bound, and ant algorithms =-=[16]-=-; the latter have some of the flavor of message passing, walking trails over the graph representing a QAP and iteratively updating scores of different assignments to the QAP. To the best of our knowle... |

55 | Digital tapestry
- Rother, Kumar, et al.
- 2005
(Show Context)
Citation Context ...ributes both to its improved convergence and to the better results it obtains even without convergence. Some very recent work explores the case where a regular MRF contains terms that are not regular =-=[14, 13]-=-, but this work is largely specific to certain types of “close-to-regular” MRFs. It would be interesting to compare COMPOSE and these methods on a range of networks containing regular subgraphs. Our ... |

54 | On optimality of tree-reweighted maxproduct message-passing
- Kolmogorov, Wainwright
- 2005
(Show Context)
Citation Context ...h of the subnetworks is computed exactly using a black-box subroutine. We note that this message passing scheme is somewhat related to the tree-reweighted maxproduct (TRW) method of Wainwright et al. =-=[8]-=-, where the network distribution is partitioned as a weighted combination of trees, which also communicate pseudo-max-marginals with each other.s4 Efficient Computation of Max-Marginals In this sectio... |

39 | Measuring uncertainty in graph cut solutions
- Kohli, Torr
- 2008
(Show Context)
Citation Context ...ed to find a solution with guaranteed approximation error relative to the optimal MAP assignment. As above, a single joint assignment does not suffice for our purposes. In recent work, Kohli and Torr =-=[7]-=-, studying the problem of confidence estimation in MAP problems, showed how all of the max-marginals in a regular network can be computed using dynamic algorithms for flow computations. Their method a... |

32 | A scheme for unifying optimization and constraint satisfaction methods
- Hooker, Ottosson, et al.
(Show Context)
Citation Context ..., or other efficient approaches. For example, the constraint satisfaction community has studied several special-purpose constraint types that can be solved more efficiently than using generic methods =-=[4]-=-; it would be interesting to explore whether these constraints arise within MRFs, and, if so, whether the special-purpose procedures can be integrated into the COMPOSE framework. Overall, we believe t... |

9 | MRF’s for MRI’s: Bayesian reconstruction of MR images via graph cuts
- Raj, Singh, et al.
(Show Context)
Citation Context ...nts between values of adjacent variables, can be solved using matching algorithms. These types of networks have been shown to be applicable in a variety of applications, such as stereo reconstruction =-=[13]-=- and segmentation for regular networks, and image correspondence [15] or word alignment for matching networks [19].sIn many real-world applications, however, the problem formulation does not fall neat... |

2 |
Residual belief propagation
- Elidan, McGraw, et al.
- 2006
(Show Context)
Citation Context .... In this case, the geometric constraints were more elaborate, and it was not clear how to construct a good set of spanning trees. We therefore used a variant on AMP called residual max-product (RMP) =-=[3]-=- that schedules messages in an informed way over the network; in this work and others, we have found this variant to achieve better performance than TRMP on difficult networks. Fig. 2(a) shows a sourc... |