Results 1  10
of
48
libDAI: A free/open source C++ library for discrete approximate inference methods
, 2008
"... This paper describes the software package libDAI, a free & open source C++ library that provides implementations of various exact and approximate inference methods for graphical models with discretevalued variables. libDAI supports directed graphical models (Bayesian networks) as well as undire ..."
Abstract

Cited by 72 (1 self)
 Add to MetaCart
This paper describes the software package libDAI, a free & open source C++ library that provides implementations of various exact and approximate inference methods for graphical models with discretevalued variables. libDAI supports directed graphical models (Bayesian networks) as well as undirected ones (Markov random fields and factor graphs). It offers various approximations of the partition sum, marginal probability distributions and maximum probability states. Parameter learning is also supported. A feature comparison with other open source software packages for approximate inference is given. libDAI is licensed under the GPL v2+ license and is available at
An Alternating Direction Method for Dual MAP LP Relaxation
"... Abstract. Maximum aposteriori (MAP) estimation is an important task in many applications of probabilistic graphical models. Although finding an exact solution is generally intractable, approximations based on linear programming (LP) relaxation often provide good approximate solutions. In this paper ..."
Abstract

Cited by 33 (3 self)
 Add to MetaCart
Abstract. Maximum aposteriori (MAP) estimation is an important task in many applications of probabilistic graphical models. Although finding an exact solution is generally intractable, approximations based on linear programming (LP) relaxation often provide good approximate solutions. In this paper we present an algorithm for solving the LP relaxation optimization problem. In order to overcome the lack of strict convexity, we apply an augmented Lagrangian method to the dual LP. The algorithm, based on the alternating direction method of multipliers (ADMM), is guaranteed to converge to the global optimum of the LP relaxation objective. Our experimental results show that this algorithm is competitive with other stateoftheart algorithms for approximate MAP estimation.
Learning Efficiently with Approximate Inference via Dual Losses
"... Many structured prediction tasks involve complex models where inference is computationally intractable, but where it can be well approximated using a linear programming relaxation. Previous approaches for learning for structured prediction (e.g., cuttingplane, subgradient methods, perceptron) repeat ..."
Abstract

Cited by 37 (8 self)
 Add to MetaCart
Many structured prediction tasks involve complex models where inference is computationally intractable, but where it can be well approximated using a linear programming relaxation. Previous approaches for learning for structured prediction (e.g., cuttingplane, subgradient methods, perceptron) repeatedly make predictions for some of the data points. These approaches are computationally demanding because each prediction involves solving a linear program to optimality. We present a scalable algorithm for learning for structured prediction. The main idea is to instead solve the dual of the structured prediction loss. We formulate the learning task as a convex minimization over both the weights and the dual variables corresponding to each data point. As a result, we can begin to optimize the weights even before completely solving any of the individual prediction problems. We show how the dual variables can be efficiently optimized using coordinate descent. Our algorithm is competitive with stateoftheart methods such as stochastic subgradient and cuttingplane. 1.
FastInf: An efficient approximate inference library
 Journal of Machine Learning Research
"... The FastInf C++ library is designed to perform memory and time efficient approximate inference in largescale discrete undirected graphical models. The focus of the library is propagation based approximate inference methods, ranging from the basic loopy belief propagation algorithm to propagation ba ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
The FastInf C++ library is designed to perform memory and time efficient approximate inference in largescale discrete undirected graphical models. The focus of the library is propagation based approximate inference methods, ranging from the basic loopy belief propagation algorithm to propagation based on convex free energies. Various message scheduling schemes that improve on the standard synchronous or asynchronous approaches are included. Also implemented are a clique tree based exact inference, Gibbs sampling, and the mean field algorithm. In addition to inference, FastInf provides parameter estimation capabilities as well as representation and learning of shared parameters. It offers a rich interface that facilitates extension of the basic classes to other inference and learning methods.
‖w‖2 + 1 M
"... We have shown that with our enhanced representation, the ranking problem for given weights w reduces to the one in Joachims (2005) in the case of a fullyfactored model. Here we show a similar result for the learning problem. Recall that our learning objective is defined as: min ..."
Abstract
 Add to MetaCart
We have shown that with our enhanced representation, the ranking problem for given weights w reduces to the one in Joachims (2005) in the case of a fullyfactored model. Here we show a similar result for the learning problem. Recall that our learning objective is defined as: min
‖w‖2 + 1 M
"... We have shown that with our enhanced representation, the ranking problem for given weights w reduces to the one in Joachims (2005) in the case of a fullyfactored model. Here we show a similar result for the learning problem. Recall that our learning objective is defined as: min ..."
Abstract
 Add to MetaCart
We have shown that with our enhanced representation, the ranking problem for given weights w reduces to the one in Joachims (2005) in the case of a fullyfactored model. Here we show a similar result for the learning problem. Recall that our learning objective is defined as: min
Convergence rate analysis of MAP coordinate minimization algorithms
 In NIPS. 2012
"... Finding maximum a posteriori (MAP) assignments in graphical models is an important task in many applications. Since the problem is generally hard, linear programming (LP) relaxations are often used. Solving these relaxations efficiently is thus an important practical problem. In recent years, seve ..."
Abstract

Cited by 11 (3 self)
 Add to MetaCart
Finding maximum a posteriori (MAP) assignments in graphical models is an important task in many applications. Since the problem is generally hard, linear programming (LP) relaxations are often used. Solving these relaxations efficiently is thus an important practical problem. In recent years, several authors have proposed message passing updates corresponding to coordinate descent in the dual LP. However, these are generally not guaranteed to converge to a global optimum. One approach to remedy this is to smooth the LP, and perform coordinate descent on the smoothed dual. However, little is known about the convergence rate of this procedure. Here we perform a thorough rate analysis of such schemes and derive primal and dual convergence rates. We also provide a simple dual to primal mapping that yields feasible primal solutions with a guaranteed rate of convergence. Empirical evaluation supports our theoretical claims and shows that the method is highly competitive with state of the art approaches that yield global optima. 1
Convexifying the bethe free energy
 in Conference on Uncertainty in Artifical Intelligence (UAI
, 2009
"... The introduction of loopy belief propagation (LBP) revitalized the application of graphical models in many domains. Many recent works present improvements on the basic LBP algorithm in an attempt to overcome convergence and local optima problems. Notable among these are convexified free energy appro ..."
Abstract

Cited by 15 (2 self)
 Add to MetaCart
The introduction of loopy belief propagation (LBP) revitalized the application of graphical models in many domains. Many recent works present improvements on the basic LBP algorithm in an attempt to overcome convergence and local optima problems. Notable among these are convexified free energy approximations that lead to inference procedures with provable convergence and quality properties. However, empirically LBP still outperforms most of its convex variants in a variety of settings, as we also demonstrate here. Motivated by this fact we seek convexified free energies that directly approximate the Bethe free energy. We show that the proposed approximations compare favorably with stateofthe art convex free energy approximations. 1
Learning Structured Models with the AUC Loss and Its Generalizations
"... Many problems involve the prediction of multiple, possibly dependent labels. The structured output prediction framework builds predictors that take these dependencies into account and use them to improve accuracy. In many such tasks, performance is evaluated by the Area Under the ROC Curve (AUC). ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Many problems involve the prediction of multiple, possibly dependent labels. The structured output prediction framework builds predictors that take these dependencies into account and use them to improve accuracy. In many such tasks, performance is evaluated by the Area Under the ROC Curve (AUC). While a framework for optimizing the AUC loss for unstructured models exists, it does not naturally extend to structured models. In this work, we propose a representation and learning formulation for optimizing structured models over the AUC loss, show how our approach generalizes the unstructured case, and provide algorithms for solving the resulting inference and learning problems. We also explore several new variants of the AUC measure which naturally arise from our formulation. Finally, we empirically show the utility of our approach in several domains. 1
ContinuousTime Belief Propagation
, 2010
"... Many temporal processes can be naturally modeled as a stochastic system that evolves continuously over time. The representation language of continuoustime Bayesian networks allows to succinctly describe multicomponent continuoustime stochastic processes. A crucial element in applications of such ..."
Abstract

Cited by 14 (3 self)
 Add to MetaCart
Many temporal processes can be naturally modeled as a stochastic system that evolves continuously over time. The representation language of continuoustime Bayesian networks allows to succinctly describe multicomponent continuoustime stochastic processes. A crucial element in applications of such models is inference. Here we introduce a variational approximation scheme, which is a natural extension of Belief Propagation for continuoustime processes. In this scheme, we view messages as inhomogeneous Markov processes over individual components. This leads to a relatively simple procedure that allows to easily incorporate adaptive ordinary differential equation (ODE) solvers to perform individual steps. We provide the theoretical foundations for the approximation, and show how it performs on a range of networks. Our results demonstrate that our method is quite accurate on singly connected networks, and provides close approximations in more complex ones.
Results 1  10
of
48