Results 1  10
of
10
Scaling MPE Inference for Constrained Continuous Markov Random Fields with Consensus Optimization
"... Probabilistic graphical models are powerful tools for analyzing constrained, continuous domains. However, finding mostprobable explanations (MPEs) in these models can be computationally expensive. In this paper, we improve the scalability of MPE inference in a class of graphical models with piecewi ..."
Abstract

Cited by 8 (8 self)
 Add to MetaCart
Probabilistic graphical models are powerful tools for analyzing constrained, continuous domains. However, finding mostprobable explanations (MPEs) in these models can be computationally expensive. In this paper, we improve the scalability of MPE inference in a class of graphical models with piecewiselinear and piecewisequadratic dependencies and linear constraints over continuous domains. We derive algorithms based on a consensusoptimization framework and demonstrate their superior performance over state of the art. We show empirically that in a largescale voterpreference modeling problem our algorithms scale linearly in the number of dependencies and constraints. 1
Hingeloss Markov Random Fields: Convex Inference for Structured Prediction
 In Uncertainty in Artificial Intelligence
, 2013
"... Graphical models for structured domains are powerful tools, but the computational complexities of combinatorial prediction spaces can force restrictions on models, or require approximate inference in order to be tractable. Instead of working in a combinatorial space, we use hingeloss Markov random ..."
Abstract

Cited by 4 (4 self)
 Add to MetaCart
Graphical models for structured domains are powerful tools, but the computational complexities of combinatorial prediction spaces can force restrictions on models, or require approximate inference in order to be tractable. Instead of working in a combinatorial space, we use hingeloss Markov random fields (HLMRFs), an expressive class of graphical models with logconcave density functions over continuous variables, which can represent confidences in discrete predictions. This paper demonstrates that HLMRFs are general tools for fast and accurate structured prediction. We introduce the first inference algorithm that is both scalable and applicable to the full class of HLMRFs, and show how to train HLMRFs with several learning algorithms. Our experiments show that HLMRFs match or surpass the predictive performance of stateoftheart methods, including discrete models, in four application domains. 1
Efficiently Searching for Frustrated Cycles in MAP Inference
"... Dual decomposition provides a tractable framework for designing algorithms for finding the most probable (MAP) configuration in graphical models. However, for many realworld inference problems, the typical decomposition has a large integrality gap, due to frustrated cycles. One way to tighten the r ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Dual decomposition provides a tractable framework for designing algorithms for finding the most probable (MAP) configuration in graphical models. However, for many realworld inference problems, the typical decomposition has a large integrality gap, due to frustrated cycles. One way to tighten the relaxation is to introduce additional constraints that explicitly enforce cycle consistency. Earlier work showed that clusterpursuit algorithms, which iteratively introduce cycle and other higherorder consistency constraints, allows one to exactly solve many hard inference problems. However, these algorithms explicitly enumerate a candidate set of clusters, limiting them to triplets or other short cycles. We solve the search problem for cycle constraints, giving a nearly linear time algorithm for finding the most frustrated cycle of arbitrary length. We show how to use this search algorithm together with the dual decomposition framework and clusterpursuit. The new algorithm exactly solves MAP inference problems arising from relational classification and stereo vision. 1
Convergence Rate Analysis of MAP Coordinate Minimization Algorithms
"... Finding maximum a posteriori (MAP) assignments in graphical models is an important task in many applications. Since the problem is generally hard, linear programming (LP) relaxations are often used. Solving these relaxations efficiently is thus an important practical problem. In recent years, severa ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Finding maximum a posteriori (MAP) assignments in graphical models is an important task in many applications. Since the problem is generally hard, linear programming (LP) relaxations are often used. Solving these relaxations efficiently is thus an important practical problem. In recent years, several authors have proposed message passing updates corresponding to coordinate descent in the dual LP. However, these are generally not guaranteed to converge to a global optimum. One approach to remedy this is to smooth the LP, and perform coordinate descent on the smoothed dual. However, little is known about the convergence rate of this procedure. Here we perform a thorough rate analysis of such schemes and derive primal and dual convergence rates. We also provide a simple dual to primal mapping that yields feasible primal solutions with a guaranteed rate of convergence. Empirical evaluation supports our theoretical claims and shows that the method is highly competitive with state of the art approaches that yield global optima. 1
Tractable SemiSupervised Learning of Complex Structured Prediction Models
"... Abstract. Semisupervised learning has been widely studied in the literature. However, most previous works assume that the output structure is simple enough to allow the direct use of tractable inference/learning algorithms (e.g., binary label or linear chain). Therefore, these methods cannot be app ..."
Abstract
 Add to MetaCart
Abstract. Semisupervised learning has been widely studied in the literature. However, most previous works assume that the output structure is simple enough to allow the direct use of tractable inference/learning algorithms (e.g., binary label or linear chain). Therefore, these methods cannot be applied to problems with complex structure. In this paper, we propose an approximate semisupervised learning method that uses piecewise training for estimating the model weights and a dual decomposition approach for solving the inference problem of finding the labels of unlabeled data subject to domain specific constraints. This allows us to extend semisupervised learning to general structured prediction problems. As an example, we apply this approach to the problem of multilabel classification (a fully connected pairwise Markov random field). Experimental results on benchmark data show that, in spite of using approximations, the approach is effective and yields good improvements in generalization performance over the plain supervised method. In addition, we demonstrate that our inference engine can be applied to other semisupervised learning frameworks, and extends them to solve problems with complex structure. 1
Solving Combinatorial Optimization Problems using Relaxed Linear Programming: A High Performance Computing Perspective
"... Several important combinatorial optimization problems can be formulated as maximum a posteriori (MAP) inference in discrete graphical models. We adopt the recently proposed parallel MAP inference algorithm BetheADMM and implement it using message passing interface (MPI) to fully utilize the computi ..."
Abstract
 Add to MetaCart
Several important combinatorial optimization problems can be formulated as maximum a posteriori (MAP) inference in discrete graphical models. We adopt the recently proposed parallel MAP inference algorithm BetheADMM and implement it using message passing interface (MPI) to fully utilize the computing power provided by the modern supercomputers with thousands of cores. The empirical results show that our parallel implementation scales almost linearly even with thousands of cores. Keywords alternating direction method of multipliers, Markov random field, maximum a posteriori inference, message passing interface 1.
BetheADMM for Tree Decomposition based Parallel MAP Inference
"... We consider the problem of maximum a posteriori (MAP) inference in discrete graphical models. We present a parallel MAP inference algorithm called BetheADMM based on two ideas: treedecomposition of the graph and the alternating direction method of multipliers (ADMM). However, unlike the standard A ..."
Abstract
 Add to MetaCart
We consider the problem of maximum a posteriori (MAP) inference in discrete graphical models. We present a parallel MAP inference algorithm called BetheADMM based on two ideas: treedecomposition of the graph and the alternating direction method of multipliers (ADMM). However, unlike the standard ADMM, we use an inexact ADMM augmented with a Bethedivergence based proximal function, which makes each subproblem in ADMM easy to solve in parallel using the sumproduct algorithm. We rigorously prove global convergence of BetheADMM. The proposed algorithm is extensively evaluated on both synthetic and real datasets to illustrate its effectiveness. Further, the parallel BetheADMM is shown to scale almost linearly with increasing number of cores. 1
Globally Convergent Dual MAP LP Relaxation Solvers using FenchelYoung Margins
"... While finding the exact solution for the MAP inference problem is intractable for many realworld tasks, MAP LP relaxations have been shown to be very effective in practice. However, the most efficient methods that perform block coordinate descent can get stuck in suboptimal points as they are not ..."
Abstract
 Add to MetaCart
While finding the exact solution for the MAP inference problem is intractable for many realworld tasks, MAP LP relaxations have been shown to be very effective in practice. However, the most efficient methods that perform block coordinate descent can get stuck in suboptimal points as they are not globally convergent. In this work we propose to augment these algorithms with an ɛdescent approach and present a method to efficiently optimize for a descent direction in the subdifferential using a marginbased formulation of the FenchelYoung duality theorem. Furthermore, the presented approach provides a methodology to construct a primal optimal solution from its dual optimal counterpart. We demonstrate the efficiency of the presented approach on spin glass models and protein interaction problems and show that our approach outperforms stateoftheart solvers. 1
DARWIN: A Framework for Machine Learning and Computer Vision Research and Development
"... We present an opensource platformindependent C++ framework for machine learning and computer vision research. The framework includes a wide range of standard machine learning and graphical models algorithms as well as reference implementations for many machine learning and computer vision applicat ..."
Abstract
 Add to MetaCart
We present an opensource platformindependent C++ framework for machine learning and computer vision research. The framework includes a wide range of standard machine learning and graphical models algorithms as well as reference implementations for many machine learning and computer vision applications. The framework contains Matlab wrappers for core components of the library and an experimental graphical user interface for developing and visualizing machine learning data flows.