Results 1  10
of
49
Complexity results and approximation strategies for map explanations
 Journal of Artificial Intelligence Research
, 2004
"... MAP is the problem of finding a most probable instantiation of a set of variables given evidence. MAP has always been perceived to be significantly harder than the related problems of computing the probability of a variable instantiation (Pr), or the problem of computing the most probable explanatio ..."
Abstract

Cited by 43 (3 self)
 Add to MetaCart
(Show Context)
MAP is the problem of finding a most probable instantiation of a set of variables given evidence. MAP has always been perceived to be significantly harder than the related problems of computing the probability of a variable instantiation (Pr), or the problem of computing the most probable explanation (MPE). This paper investigates the complexity of MAP in Bayesian networks. Specifically, we show that MAP is complete for NP PP and provide further negative complexity results for algorithms based on variable elimination. We also show that MAP remains hard even when MPE and Pr become easy. For example, we show that MAP is NPcomplete when the networks are restricted to polytrees, and even then can not be effectively approximated. Given the difficulty of computing MAP exactly, and the difficulty of approximating MAP while providing useful guarantees on the resulting approximation, we investigate best effort approximations. We introduce a generic MAP approximation framework. We provide two instantiations of the framework; one for networks which are amenable to exact inference (Pr), and one for networks for which even exact inference is too hard. This allows MAP approximation on networks that are too complex to even exactly solve the easier problems, Pr and MPE. Experimental results indicate that using these approximation algorithms provides much better solutions than standard techniques, and provide accurate MAP estimates in many cases. 1.
AND/OR branchandbound search for combinatorial optimization in graphical models
, 2008
"... We introduce a new generation of depthfirst BranchandBound algorithms that explore the AND/OR search tree using static and dynamic variable orderings for solving general constraint optimization problems. The virtue of the AND/OR representation of the search space is that its size may be far small ..."
Abstract

Cited by 39 (19 self)
 Add to MetaCart
We introduce a new generation of depthfirst BranchandBound algorithms that explore the AND/OR search tree using static and dynamic variable orderings for solving general constraint optimization problems. The virtue of the AND/OR representation of the search space is that its size may be far smaller than that of a traditional OR representation, which can translate into significant time savings for search algorithms. The focus of this paper is on linear space search which explores the AND/OR search tree rather than the search graph and therefore make no attempt to cache information. We investigate the power of the minibucket heuristics within the AND/OR search space, in both static and dynamic setups. We focus on two most common optimization problems in graphical models: finding the Most Probable Explanation (MPE) in Bayesian networks and solving Weighted CSPs (WCSP). In extensive empirical evaluations we demonstrate that the new AND/OR BranchandBound approach improves considerably over the traditional OR search strategy and show how various variable ordering schemes impact the performance of the AND/OR search scheme.
Open constraint programming
 Artifitial Intelligence
"... Constraint satisfaction and optimization problems often involve multiple participants. For example, producing an automobile involves a supply chain of many companies. Scheduling production, delivery and assembly of the different parts would best be solved as a constraint optimization problem ([35]). ..."
Abstract

Cited by 37 (5 self)
 Add to MetaCart
(Show Context)
Constraint satisfaction and optimization problems often involve multiple participants. For example, producing an automobile involves a supply chain of many companies. Scheduling production, delivery and assembly of the different parts would best be solved as a constraint optimization problem ([35]). A more familiar task for most of us is meeting scheduling: arrange a set of meetings with varying participants such that no two meetings involving the same person are scheduled at the same time, while respecting order and deadline constraints ([18, 22]). Another application that has been studied in detail is coordinating a network of distributed sensors ([2]). Such problems can of course be solved by gathering all constraints and optimization criteria into a single large CSP, and then solving this problem using a centralized algorithm. In practice there are many cases where this is not feasible, because it is impossible to bound the problem to a manageable set of variables. For example, in meeting scheduling, once two people are planning a common meeting, this meeting is potentially in conflict with many other meetings either of them are planning and whose times are decided in parallel. A centralized solver does not know beforehand
Solving MAP exactly using systematic search
 In Proceedings of the 19th Conference on Uncertainty in Artificial Intelligence (UAI–03
"... MAP is the problem of finding a most probable instantiation of a set of variables in a Bayesian network, given some partial evidence about the complement of that set. Unlike posterior probabilities, or MPE (a special case of MAP), the time and space complexity of structure–based algorithms for MAP a ..."
Abstract

Cited by 29 (1 self)
 Add to MetaCart
MAP is the problem of finding a most probable instantiation of a set of variables in a Bayesian network, given some partial evidence about the complement of that set. Unlike posterior probabilities, or MPE (a special case of MAP), the time and space complexity of structure–based algorithms for MAP are not only exponential in the network treewidth, but in a larger parameter known as the constrained treewidth. In practice, this means that computing MAP can be orders of magnitude more expensive than computing posterior probabilities or MPE. We introduce in this paper a new, simple upper bound on the probability of a MAP solution, which is shown to be generally much tighter than existing bounds. We then use the proposed upper bound to develop a branch–and– bound search algorithm for solving MAP exactly. Experimental results demonstrate that the search algorithm is able to solve many problems that are far beyond the reach of any structure–based method for MAP. For example, we show that the proposed algorithm can compute MAP exactly and efficiently for some networks whose constrained treewidth is more than 40. 1
JoinGraph Propagation Algorithms
"... The paper investigates parameterized approximate messagepassing schemes that are based on bounded inference and are inspired by Pearl’s belief propagation algorithm (BP). We start with the bounded inference miniclustering algorithm and then move to the iterative scheme called Iterative JoinGraph ..."
Abstract

Cited by 18 (8 self)
 Add to MetaCart
(Show Context)
The paper investigates parameterized approximate messagepassing schemes that are based on bounded inference and are inspired by Pearl’s belief propagation algorithm (BP). We start with the bounded inference miniclustering algorithm and then move to the iterative scheme called Iterative JoinGraph Propagation (IJGP), that combines both iteration and bounded inference. The algorithm IJGP belongs to the class of Generalized Belief Propagation algorithms, a framework that allowed connections with approximate algorithms from statistical physics and is shown empirically to surpass the performance of miniclustering and belief propagation, as well as a number of other stateoftheart algorithms on several classes of networks. We also provide insight into the accuracy of IBP and IJGP by relating these algorithms to well known classes of constraint propagation schemes.
Ordered valuation algebras: a generic framework for approximating inference
 International Journal of Approximate Reasoning
, 2004
"... The paper presents a generic approach of approximating inference. The method is based on the concept of valuation algebras with its wide range of possible applications in many different domains. We present convenient resourcebounded anytime algorithms, where the maximal time of computation is deter ..."
Abstract

Cited by 17 (1 self)
 Add to MetaCart
(Show Context)
The paper presents a generic approach of approximating inference. The method is based on the concept of valuation algebras with its wide range of possible applications in many different domains. We present convenient resourcebounded anytime algorithms, where the maximal time of computation is determined by the user. Key words: Approximation, anytime algorithms, resourcebounded computation, valuation algebras, local computation, binary join trees, bucket elimination, minibuckets. 1
Memory intensive branchandbound search for graphical models
 In AAAI
, 2006
"... AND/OR search spaces have recently been introduced as a unifying paradigm for advanced algorithmic schemes for graphical models. The main virtue of this representation is its sensitivity to the structure of the model, which can translate into exponential time savings for search algorithms. AND/OR Br ..."
Abstract

Cited by 16 (7 self)
 Add to MetaCart
(Show Context)
AND/OR search spaces have recently been introduced as a unifying paradigm for advanced algorithmic schemes for graphical models. The main virtue of this representation is its sensitivity to the structure of the model, which can translate into exponential time savings for search algorithms. AND/OR BranchandBound (AOBB) is a new algorithm that explores the AND/OR search tree for solving optimization tasks in graphical models. In this paper we extend the algorithm to explore an AND/OR search graph by equipping it with a contextbased adaptive caching scheme similar to good and nogood recording. The efficiency of the new graph search algorithm is demonstrated empirically on various benchmarks, including the very challenging ones that arise in genetic linkage analysis.
Memory Intensive AND/OR Search for Combinatorial Optimization in Graphical Models
"... In this paper we explore the impact of caching on search in the context of the recent framework of AND/OR search in graphical models. Specifically, we extend the depthfirst AND/OR BranchandBound tree search algorithm to explore an AND/OR search graph by equipping it with an adaptive caching schem ..."
Abstract

Cited by 15 (9 self)
 Add to MetaCart
(Show Context)
In this paper we explore the impact of caching on search in the context of the recent framework of AND/OR search in graphical models. Specifically, we extend the depthfirst AND/OR BranchandBound tree search algorithm to explore an AND/OR search graph by equipping it with an adaptive caching scheme similar to good and nogood recording. Furthermore, we present bestfirst search algorithms for traversing the same underlying AND/OR search graph and compare both depthfirst and bestfirst approaches empirically. We focus on two common optimization problems in graphical models: finding the Most Probable Explanation (MPE) in belief networks and solving Weighted CSPs (WCSP). In an extensive empirical evaluation we demonstrate conclusively the superiority of the memory intensive AND/OR search algorithms on a variety of benchmarks including random and realworld problem instances.
Pushing the power of stochastic greedy ordering schemes for inference in graphical models
 IN AAAI 2011
, 2011
"... We study iterative randomized greedy algorithms for generating (elimination) orderings with small induced width and state space size two parameters known to bound the complexity of inference in graphical models. We propose and implement the Iterative Greedy Variable Ordering (IGVO) algorithm, a new ..."
Abstract

Cited by 15 (10 self)
 Add to MetaCart
We study iterative randomized greedy algorithms for generating (elimination) orderings with small induced width and state space size two parameters known to bound the complexity of inference in graphical models. We propose and implement the Iterative Greedy Variable Ordering (IGVO) algorithm, a new variant within this algorithm class. An empirical evaluation using different ranking functions and conditions of randomness, demonstrates that IGVO finds significantly better orderings than standard greedy ordering implementations when evaluated within an anytime framework. Additional order of magnitude improvements are demonstrated on a multicore system, thus further expanding the set of solvable graphical models. The experiments also confirm the superiority of the MinFill heuristic within the iterative scheme.
Systematic vs. Nonsystematic Algorithms for Solving the MPE Task
 IN PROCEEDINGS OF UNCERTAINTY IN ARTIFICIAL INTELLIGENCE (UAI
, 2003
"... The paper explores the power of two systematic Branch and Bound search algorithms that exploit partitionbased heuristics, BBBT (a new algorithm for which the heuristic information is constructed during search and allows dynamic variable/value ordering) and its predecessor BBMB (for which the ..."
Abstract

Cited by 13 (2 self)
 Add to MetaCart
The paper explores the power of two systematic Branch and Bound search algorithms that exploit partitionbased heuristics, BBBT (a new algorithm for which the heuristic information is constructed during search and allows dynamic variable/value ordering) and its predecessor BBMB (for which the heuristic information is precompiled) and compares them against a number of popular local search algorithms for the MPE problem as well as against the recently popular iterative belief propagation algorithms. We show empirically that the new Branch and Bound algorithm, BBBT demonstrates tremendous pruning of the search space far beyond its predecessor, BBMB which translates to impressive time saving for some classes of problems. Second, when viewed as approximation schemes, BBBT/BBMB together are highly competitive with the best known SLS algorithms and are superior, especially when the domain sizes increase beyond 2. The