Results 1  10
of
18
Scaling MPE Inference for Constrained Continuous Markov Random Fields with Consensus Optimization
"... Probabilistic graphical models are powerful tools for analyzing constrained, continuous domains. However, finding mostprobable explanations (MPEs) in these models can be computationally expensive. In this paper, we improve the scalability of MPE inference in a class of graphical models with piecewi ..."
Abstract

Cited by 12 (12 self)
 Add to MetaCart
(Show Context)
Probabilistic graphical models are powerful tools for analyzing constrained, continuous domains. However, finding mostprobable explanations (MPEs) in these models can be computationally expensive. In this paper, we improve the scalability of MPE inference in a class of graphical models with piecewiselinear and piecewisequadratic dependencies and linear constraints over continuous domains. We derive algorithms based on a consensusoptimization framework and demonstrate their superior performance over state of the art. We show empirically that in a largescale voterpreference modeling problem our algorithms scale linearly in the number of dependencies and constraints. 1
Hingeloss Markov Random Fields: Convex Inference for Structured Prediction
 In Uncertainty in Artificial Intelligence
, 2013
"... Graphical models for structured domains are powerful tools, but the computational complexities of combinatorial prediction spaces can force restrictions on models, or require approximate inference in order to be tractable. Instead of working in a combinatorial space, we use hingeloss Markov random ..."
Abstract

Cited by 10 (10 self)
 Add to MetaCart
(Show Context)
Graphical models for structured domains are powerful tools, but the computational complexities of combinatorial prediction spaces can force restrictions on models, or require approximate inference in order to be tractable. Instead of working in a combinatorial space, we use hingeloss Markov random fields (HLMRFs), an expressive class of graphical models with logconcave density functions over continuous variables, which can represent confidences in discrete predictions. This paper demonstrates that HLMRFs are general tools for fast and accurate structured prediction. We introduce the first inference algorithm that is both scalable and applicable to the full class of HLMRFs, and show how to train HLMRFs with several learning algorithms. Our experiments show that HLMRFs match or surpass the predictive performance of stateoftheart methods, including discrete models, in four application domains. 1
A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing
"... Dual decomposition, and more generally Lagrangian relaxation, is a classical method for combinatorial optimization; it has recently been applied to several inference problems in natural language processing (NLP). This tutorial gives an overview of the technique. We describe example algorithms, descr ..."
Abstract

Cited by 7 (1 self)
 Add to MetaCart
(Show Context)
Dual decomposition, and more generally Lagrangian relaxation, is a classical method for combinatorial optimization; it has recently been applied to several inference problems in natural language processing (NLP). This tutorial gives an overview of the technique. We describe example algorithms, describe formal guarantees for the method, and describe practical issues in implementing the algorithms. While our examples are predominantly drawn from the NLP literature, the material should be of general relevance to inference problems in machine learning. A central theme of this tutorial is that Lagrangian relaxation is naturally applied in conjunction with a broad class of combinatorial algorithms, allowing inference in models that go significantly beyond previous work on Lagrangian relaxation for inference in graphical models. 1.
Efficiently Searching for Frustrated Cycles in MAP Inference
"... Dual decomposition provides a tractable framework for designing algorithms for finding the most probable (MAP) configuration in graphical models. However, for many realworld inference problems, the typical decomposition has a large integrality gap, due to frustrated cycles. One way to tighten the r ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Dual decomposition provides a tractable framework for designing algorithms for finding the most probable (MAP) configuration in graphical models. However, for many realworld inference problems, the typical decomposition has a large integrality gap, due to frustrated cycles. One way to tighten the relaxation is to introduce additional constraints that explicitly enforce cycle consistency. Earlier work showed that clusterpursuit algorithms, which iteratively introduce cycle and other higherorder consistency constraints, allows one to exactly solve many hard inference problems. However, these algorithms explicitly enumerate a candidate set of clusters, limiting them to triplets or other short cycles. We solve the search problem for cycle constraints, giving a nearly linear time algorithm for finding the most frustrated cycle of arbitrary length. We show how to use this search algorithm together with the dual decomposition framework and clusterpursuit. The new algorithm exactly solves MAP inference problems arising from relational classification and stereo vision. 1
Convergence Rate Analysis of MAP Coordinate Minimization Algorithms
"... Finding maximum a posteriori (MAP) assignments in graphical models is an important task in many applications. Since the problem is generally hard, linear programming (LP) relaxations are often used. Solving these relaxations efficiently is thus an important practical problem. In recent years, severa ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Finding maximum a posteriori (MAP) assignments in graphical models is an important task in many applications. Since the problem is generally hard, linear programming (LP) relaxations are often used. Solving these relaxations efficiently is thus an important practical problem. In recent years, several authors have proposed message passing updates corresponding to coordinate descent in the dual LP. However, these are generally not guaranteed to converge to a global optimum. One approach to remedy this is to smooth the LP, and perform coordinate descent on the smoothed dual. However, little is known about the convergence rate of this procedure. Here we perform a thorough rate analysis of such schemes and derive primal and dual convergence rates. We also provide a simple dual to primal mapping that yields feasible primal solutions with a guaranteed rate of convergence. Empirical evaluation supports our theoretical claims and shows that the method is highly competitive with state of the art approaches that yield global optima. 1
BetheADMM for Tree Decomposition based Parallel MAP Inference
"... We consider the problem of maximum a posteriori (MAP) inference in discrete graphical models. We present a parallel MAP inference algorithm called BetheADMM based on two ideas: treedecomposition of the graph and the alternating direction method of multipliers (ADMM). However, unlike the standard A ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
We consider the problem of maximum a posteriori (MAP) inference in discrete graphical models. We present a parallel MAP inference algorithm called BetheADMM based on two ideas: treedecomposition of the graph and the alternating direction method of multipliers (ADMM). However, unlike the standard ADMM, we use an inexact ADMM augmented with a Bethedivergence based proximal function, which makes each subproblem in ADMM easy to solve in parallel using the sumproduct algorithm. We rigorously prove global convergence of BetheADMM. The proposed algorithm is extensively evaluated on both synthetic and real datasets to illustrate its effectiveness. Further, the parallel BetheADMM is shown to scale almost linearly with increasing number of cores. 1
Globally Convergent Dual MAP LP Relaxation Solvers using FenchelYoung Margins
"... While finding the exact solution for the MAP inference problem is intractable for many realworld tasks, MAP LP relaxations have been shown to be very effective in practice. However, the most efficient methods that perform block coordinate descent can get stuck in suboptimal points as they are not ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
While finding the exact solution for the MAP inference problem is intractable for many realworld tasks, MAP LP relaxations have been shown to be very effective in practice. However, the most efficient methods that perform block coordinate descent can get stuck in suboptimal points as they are not globally convergent. In this work we propose to augment these algorithms with an ɛdescent approach and present a method to efficiently optimize for a descent direction in the subdifferential using a marginbased formulation of the FenchelYoung duality theorem. Furthermore, the presented approach provides a methodology to construct a primal optimal solution from its dual optimal counterpart. We demonstrate the efficiency of the presented approach on spin glass models and protein interaction problems and show that our approach outperforms stateoftheart solvers. 1
Solving Combinatorial Optimization Problems using Relaxed Linear Programming: A High Performance Computing Perspective
"... Several important combinatorial optimization problems can be formulated as maximum a posteriori (MAP) inference in discrete graphical models. We adopt the recently proposed parallel MAP inference algorithm BetheADMM and implement it using message passing interface (MPI) to fully utilize the computi ..."
Abstract
 Add to MetaCart
(Show Context)
Several important combinatorial optimization problems can be formulated as maximum a posteriori (MAP) inference in discrete graphical models. We adopt the recently proposed parallel MAP inference algorithm BetheADMM and implement it using message passing interface (MPI) to fully utilize the computing power provided by the modern supercomputers with thousands of cores. The empirical results show that our parallel implementation scales almost linearly even with thousands of cores. Keywords alternating direction method of multipliers, Markov random field, maximum a posteriori inference, message passing interface 1.
Tractable SemiSupervised Learning of Complex Structured Prediction Models
"... Abstract. Semisupervised learning has been widely studied in the literature. However, most previous works assume that the output structure is simple enough to allow the direct use of tractable inference/learning algorithms (e.g., binary label or linear chain). Therefore, these methods cannot be app ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract. Semisupervised learning has been widely studied in the literature. However, most previous works assume that the output structure is simple enough to allow the direct use of tractable inference/learning algorithms (e.g., binary label or linear chain). Therefore, these methods cannot be applied to problems with complex structure. In this paper, we propose an approximate semisupervised learning method that uses piecewise training for estimating the model weights and a dual decomposition approach for solving the inference problem of finding the labels of unlabeled data subject to domain specific constraints. This allows us to extend semisupervised learning to general structured prediction problems. As an example, we apply this approach to the problem of multilabel classification (a fully connected pairwise Markov random field). Experimental results on benchmark data show that, in spite of using approximations, the approach is effective and yields good improvements in generalization performance over the plain supervised method. In addition, we demonstrate that our inference engine can be applied to other semisupervised learning frameworks, and extends them to solve problems with complex structure. 1
Linear Approximation to ADMM for MAP inference
"... Maximum a posteriori (MAP) inference is one of the fundamental inference tasks in graphical models. MAP inference is in general NPhard, making approximate methods of interest for many problems. One successful class of approximate inference algorithms is based on linear programming (LP) relaxations. ..."
Abstract
 Add to MetaCart
(Show Context)
Maximum a posteriori (MAP) inference is one of the fundamental inference tasks in graphical models. MAP inference is in general NPhard, making approximate methods of interest for many problems. One successful class of approximate inference algorithms is based on linear programming (LP) relaxations. The augmented Lagrangian method can be used to overcome a lack of strict convexity in LP relaxations, and the Alternating Direction Method of Multipliers (ADMM) provides an elegant algorithm for finding the saddle point of the augmented Lagrangian. Here we present an ADMMbased algorithm to solve the primal form of the MAPLP whose closed form updates are based on a linear approximation technique. Our technique gives efficient, closed form updates that converge to the global optimum of the LP relaxation. We compare our algorithm to two existing ADMMbased MAPLP methods, showing that our technique is faster on general, nonbinary or nonpairwise models. 1.