Results 1  10
of
49
Sound and efficient inference with probabilistic and deterministic dependencies
, 2006
"... Reasoning with both probabilistic and deterministic dependencies is important for many realworld problems, and in particular for the emerging field of statistical relational learning. However, probabilistic inference methods like MCMC or belief propagation tend to give poor results when determin ..."
Abstract

Cited by 130 (17 self)
 Add to MetaCart
Reasoning with both probabilistic and deterministic dependencies is important for many realworld problems, and in particular for the emerging field of statistical relational learning. However, probabilistic inference methods like MCMC or belief propagation tend to give poor results when deterministic or neardeterministic dependencies are present, and logical ones like satisfiability testing are inapplicable to probabilistic ones. In this paper we propose MCSAT, an inference algorithm that combines ideas from MCMC and satisfiability. MCSAT is based on Markov logic, which defines Markov networks using weighted clauses in firstorder logic. From the point of view of MCMC,MCSAT is a slice sampler with an auxiliary variable per clause, and with a satisfiabilitybased method for sampling the original variables given the auxiliary ones. From the point of view of satisfiability, MCSAT wraps a procedure around the SampleSAT uniform sampler that enables it to sample from highly nonuniform distributions over satisfying assignments. Experiments on entity resolution and collective classification problems show that MCSAT greatly outperforms Gibbs sampling and simulated tempering over a broad range of problem sizes and degrees of determinism.
Probabilistic Theorem Proving
"... Many representation schemes combining firstorder logic and probability have been proposed in recent years. Progress in unifying logical and probabilistic inference has been slower. Existing methods are mainly variants of lifted variable elimination and belief propagation, neither of which take logic ..."
Abstract

Cited by 66 (21 self)
 Add to MetaCart
(Show Context)
Many representation schemes combining firstorder logic and probability have been proposed in recent years. Progress in unifying logical and probabilistic inference has been slower. Existing methods are mainly variants of lifted variable elimination and belief propagation, neither of which take logical structure into account. We propose the first method that has the full power of both graphical model inference and firstorder theorem proving (in finite domains with Herbrand interpretations). We first define probabilistic theorem proving, their generalization, as the problem of computing the probability of a logical formula given the probabilities or weights of a set of formulas. We then show how this can be reduced to the problem of lifted weighted model counting, and develop an efficient algorithm for the latter. We prove the correctness of this algorithm, investigate its properties, and show how it generalizes previous approaches. Experiments show that it greatly outperforms lifted variable elimination when logical structure is present. Finally, we propose an algorithm for approximate probabilistic theorem proving, and show that it can greatly outperform lifted belief propagation. 1
Memoryefficient inference in relational domains
 In Proceedings of the TwentyFirst National Conference on Artificial Intelligence
, 2006
"... Propositionalization of a firstorder theory followed by satisfiability testing has proved to be a remarkably efficient approach to inference in relational domains such as planning (Kautz & Selman 1996) and verification (Jackson 2000). More recently, weighted satisfiability solvers have been use ..."
Abstract

Cited by 45 (9 self)
 Add to MetaCart
Propositionalization of a firstorder theory followed by satisfiability testing has proved to be a remarkably efficient approach to inference in relational domains such as planning (Kautz & Selman 1996) and verification (Jackson 2000). More recently, weighted satisfiability solvers have been used successfully for MPE inference in statistical relational learners (Singla & Domingos 2005). However, fully instantiating a finite firstorder theory requires memory on the order of the number of constants raised to the arity of the clauses, which significantly limits the size of domains it can be applied to. In this paper we propose LazySAT, a variation of the WalkSAT solver that avoids this blowup by taking advantage of the extreme sparseness that is typical of relational domains (i.e., only a small fraction of ground atoms are true, and most clauses are trivially satisfied). Experiments on entity resolution and planning problems show that LazySAT reduces memory usage by orders of magnitude compared to WalkSAT, while taking comparable time to run and producing the same solutions.
AND/OR branchandbound search for combinatorial optimization in graphical models
, 2008
"... We introduce a new generation of depthfirst BranchandBound algorithms that explore the AND/OR search tree using static and dynamic variable orderings for solving general constraint optimization problems. The virtue of the AND/OR representation of the search space is that its size may be far small ..."
Abstract

Cited by 39 (19 self)
 Add to MetaCart
We introduce a new generation of depthfirst BranchandBound algorithms that explore the AND/OR search tree using static and dynamic variable orderings for solving general constraint optimization problems. The virtue of the AND/OR representation of the search space is that its size may be far smaller than that of a traditional OR representation, which can translate into significant time savings for search algorithms. The focus of this paper is on linear space search which explores the AND/OR search tree rather than the search graph and therefore make no attempt to cache information. We investigate the power of the minibucket heuristics within the AND/OR search space, in both static and dynamic setups. We focus on two most common optimization problems in graphical models: finding the Most Probable Explanation (MPE) in Bayesian networks and solving Weighted CSPs (WCSP). In extensive empirical evaluations we demonstrate that the new AND/OR BranchandBound approach improves considerably over the traditional OR search strategy and show how various variable ordering schemes impact the performance of the AND/OR search scheme.
MaxSolver: An efficient exact algorithm for (weighted) maximum satisfiability
 Artificial Intelligence
, 2005
"... Artificial Intelligence, to appear Maximum Boolean satisfiability (maxSAT) is the optimization counterpart of Boolean satisfiability (SAT), in which a variable assignment is sought to satisfy the maximum number of clauses in a Boolean formula. A branch and bound algorithm based on the DavisPutnam ..."
Abstract

Cited by 38 (1 self)
 Add to MetaCart
(Show Context)
Artificial Intelligence, to appear Maximum Boolean satisfiability (maxSAT) is the optimization counterpart of Boolean satisfiability (SAT), in which a variable assignment is sought to satisfy the maximum number of clauses in a Boolean formula. A branch and bound algorithm based on the DavisPutnamLogemannLoveland procedure (DPLL) is one of the most competitive exact algorithms for solving maxSAT. In this paper, we propose and investigate a number of strategies for maxSAT. The first strategy is a set of unit propagation or unit resolution rules for maxSAT. We summarize three existing unit propagation rules and propose a new one based on a nonlinear programming formulation of maxSAT. The second strategy is an effective lower bound based on linear programming (LP). We show that the LP lower bound can be made effective as the number of clauses increases. The third strategy consists of a a binaryclause first rule and a dynamicweighting variable ordering rule, which are motivated by a thorough analysis of two existing wellknown variable orderings. Based on the analysis of these strategies, we develop an exact solver for both maxSAT and weighted maxSAT. Our experimental results on random problem instances and many instances from the maxSAT libraries show that our new solver outperforms most of the existing exact maxSAT solvers, with orders of magnitude of improvement in many cases.
Efficient stochastic local search for MPE solving
 In Proc. of IJCAI05
, 2005
"... Finding most probable explanations (MPEs) in graphical models, such as Bayesian belief networks, is a fundamental problem in reasoning under uncertainty, and much effort has been spent on developing effective algorithms for this N Phard problem. Stochastic local search (SLS) approaches to MPE solvi ..."
Abstract

Cited by 19 (1 self)
 Add to MetaCart
Finding most probable explanations (MPEs) in graphical models, such as Bayesian belief networks, is a fundamental problem in reasoning under uncertainty, and much effort has been spent on developing effective algorithms for this N Phard problem. Stochastic local search (SLS) approaches to MPE solving have previously been explored, but were found to be not competitive with stateoftheart branch & bound methods. In this work, we identify the shortcomings of earlier SLS algorithms for the MPE problem and demonstrate how these can be overcome, leading to an SLS algorithm that substantially improves the stateoftheart in solving hard networks with many variables, large domain sizes, high degree, and, most importantly, networks with high induced width. 1
Learning and Inference in WEIGHTED LOGIC WITH APPLICATION TO NATURAL LANGUAGE PROCESSING
, 2008
"... ..."
Memory Intensive AND/OR Search for Combinatorial Optimization in Graphical Models
"... In this paper we explore the impact of caching on search in the context of the recent framework of AND/OR search in graphical models. Specifically, we extend the depthfirst AND/OR BranchandBound tree search algorithm to explore an AND/OR search graph by equipping it with an adaptive caching schem ..."
Abstract

Cited by 15 (9 self)
 Add to MetaCart
(Show Context)
In this paper we explore the impact of caching on search in the context of the recent framework of AND/OR search in graphical models. Specifically, we extend the depthfirst AND/OR BranchandBound tree search algorithm to explore an AND/OR search graph by equipping it with an adaptive caching scheme similar to good and nogood recording. Furthermore, we present bestfirst search algorithms for traversing the same underlying AND/OR search graph and compare both depthfirst and bestfirst approaches empirically. We focus on two common optimization problems in graphical models: finding the Most Probable Explanation (MPE) in belief networks and solving Weighted CSPs (WCSP). In an extensive empirical evaluation we demonstrate conclusively the superiority of the memory intensive AND/OR search algorithms on a variety of benchmarks including random and realworld problem instances.
Taming the curse of dimensionality: Discrete integration by hashing and optimization
 In ICML (To appear
, 2013
"... Integration is affected by the curse of dimensionality and quickly becomes intractable as the dimensionality of the problem grows. We propose a randomized algorithm that, with high probability, gives a constantfactor approximation of a general discrete integral defined over an exponentially large s ..."
Abstract

Cited by 15 (5 self)
 Add to MetaCart
(Show Context)
Integration is affected by the curse of dimensionality and quickly becomes intractable as the dimensionality of the problem grows. We propose a randomized algorithm that, with high probability, gives a constantfactor approximation of a general discrete integral defined over an exponentially large set. This algorithm relies on solving only a small number of instances of a discrete combinatorial optimization problem subject to randomly generated parity constraints used as a hash function. As an application, we demonstrate that with a small number of MAP queries we can efficiently approximate the partition function of discrete graphical models, which can in turn be used, for instance, for marginal computation or model selection. 1.