## An Optimal Approximation Algorithm For Bayesian Inference (1997)

### Cached

### Download Links

- [www.icsi.berkeley.edu]
- [bmir.stanford.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | Artificial Intelligence |

Citations: | 48 - 2 self |

### BibTeX

@ARTICLE{Dagum97anoptimal,

author = {Paul Dagum and Michael Luby},

title = {An Optimal Approximation Algorithm For Bayesian Inference},

journal = {Artificial Intelligence},

year = {1997},

volume = {93},

pages = {1--27}

}

### Years of Citing Articles

### OpenURL

### Abstract

Approximating the inference probability Pr[X = xjE = e] in any sense, even for a single evidence node E, is NP-hard. This result holds for belief networks that are allowed to contain extreme conditional probabilities---that is, conditional probabilities arbitrarily close to 0. Nevertheless, all previous approximation algorithms have failed to approximate efficiently many inferences, even for belief networks without extreme conditional probabilities. We prove that we can approximate efficiently probabilistic inference in belief networks without extreme conditional probabilities. We construct a randomized approximation algorithm---the bounded-variance algorithm---that is a variant of the known likelihood-weighting algorithm. The bounded-variance algorithm is the first algorithm with provably fast inference approximation on all belief networks without extreme conditional probabilities. From the bounded-variance algorithm, we construct a deterministic approximation algorithm u...

### Citations

7052 |
Probabilistic Reasoning in Intelligent Systems
- Pearl
- 1988
(Show Context)
Citation Context ...ccessfully in many realworld problems in diagnosis, prediction, and forecasting (for example, papers included in [1, 2]). Various exact algorithms exist for probabilistic inference in belief networks =-=[19, 21, 27]-=-. For a few special classes of belief networks, these algorithms can be shown to compute conditional probabilities efficiently. Cooper [6], however, showed that exact probabilistic inference for gener... |

1284 |
Local Computations with Probabilities on Graphical Structures and Their Application to Expert Systems (with Discussion
- Lauritzen, Spiegelhalter
- 1988
(Show Context)
Citation Context ...ccessfully in many realworld problems in diagnosis, prediction, and forecasting (for example, papers included in [1, 2]). Various exact algorithms exist for probabilistic inference in belief networks =-=[19, 21, 27]-=-. For a few special classes of belief networks, these algorithms can be shown to compute conditional probabilities efficiently. Cooper [6], however, showed that exact probabilistic inference for gener... |

580 |
The computational complexity of probabilistic inference using bayesian belief networks
- Cooper
- 1990
(Show Context)
Citation Context ...exist for probabilistic inference in belief networks [19, 21, 27]. For a few special classes of belief networks, these algorithms can be shown to compute conditional probabilities efficiently. Cooper =-=[6]-=-, however, showed that exact probabilistic inference for general belief networks is NP-hard. Cooper's result prompted constructions of approximation algorithms for probabilistic inference that trade o... |

562 | inference using markov chain monte carlo methods
- Neal
- 1993
(Show Context)
Citation Context ...imulation [25, 26], forward simulation, [15], likelihood weighting [13, 33], and randomized-approximation schemes [3, 4, 7, 8]. Variants of these methods such as backward simulation [14], exist; Neal =-=[23]-=- provides a good overview of the theory of simulation-based algorithms. Search-based algorithms search the space of alternative instantiations to find the most probable instantiation. These methods yi... |

287 |
Bayesian Updating in Causal Probabilistic Networks by Local Computations
- Jensen, Lauritzen, et al.
- 1990
(Show Context)
Citation Context ...ccessfully in many realworld problems in diagnosis, prediction, and forecasting (for example, papers included in [1, 2]). Various exact algorithms exist for probabilistic inference in belief networks =-=[19, 21, 27]-=-. For a few special classes of belief networks, these algorithms can be shown to compute conditional probabilities efficiently. Cooper [6], however, showed that exact probabilistic inference for gener... |

254 | Approximating Probabilistic Inference in Bayesian Belief Networks
- Dagum
- 1993
(Show Context)
Citation Context ...e random samples of the solution space. Simulation-based algorithms include straight simulation [25, 26], forward simulation, [15], likelihood weighting [13, 33], and randomized-approximation schemes =-=[3, 4, 7, 8]-=-. Variants of these methods such as backward simulation [14], exist; Neal [23] provides a good overview of the theory of simulation-based algorithms. Search-based algorithms search the space of altern... |

184 |
Propagating uncertainty in Bayesian networks by probabilistic logic sampling
- Henrion
- 1988
(Show Context)
Citation Context ...ations. Simulation-based algorithms use a source of random bits to generate random samples of the solution space. Simulation-based algorithms include straight simulation [25, 26], forward simulation, =-=[15]-=-, likelihood weighting [13, 33], and randomized-approximation schemes [3, 4, 7, 8]. Variants of these methods such as backward simulation [14], exist; Neal [23] provides a good overview of the theory ... |

184 |
Separating the polynomial-time hierarchy by oracles
- YAO
(Show Context)
Citation Context ...al-time approximations. These problems constitute the complexity class RP. Whether we can also generate deterministic approximations in polynomial time for problems in RP is a major open problem. Yao =-=[35]-=- shows that, 5 if pseudorandom generators exist, then we can generate deterministic approximations for any problem in RP in subexponential time 2 (log n) d for some integer d ? 1. Constructions of det... |

156 |
Simulation approaches to general probabilistic inference on belief networks
- Shachter, Peot
- 1990
(Show Context)
Citation Context ...gorithms use a source of random bits to generate random samples of the solution space. Simulation-based algorithms include straight simulation [25, 26], forward simulation, [15], likelihood weighting =-=[13, 33]-=-, and randomized-approximation schemes [3, 4, 7, 8]. Variants of these methods such as backward simulation [14], exist; Neal [23] provides a good overview of the theory of simulation-based algorithms.... |

100 |
Weighing and integrating evidence for stochastic simulation in bayesian networks
- Fung, Chang
- 1990
(Show Context)
Citation Context ...gorithms use a source of random bits to generate random samples of the solution space. Simulation-based algorithms include straight simulation [25, 26], forward simulation, [15], likelihood weighting =-=[13, 33]-=-, and randomized-approximation schemes [3, 4, 7, 8]. Variants of these methods such as backward simulation [14], exist; Neal [23] provides a good overview of the theory of simulation-based algorithms.... |

93 |
Evidential reasoning using stochastic simulation of causal models
- Pearl
- 1987
(Show Context)
Citation Context ...based and searchbased approximations. Simulation-based algorithms use a source of random bits to generate random samples of the solution space. Simulation-based algorithms include straight simulation =-=[25, 26]-=-, forward simulation, [15], likelihood weighting [13, 33], and randomized-approximation schemes [3, 4, 7, 8]. Variants of these methods such as backward simulation [14], exist; Neal [23] provides a go... |

92 |
Bounded Conditioning: Flexible Inference for Decision under Scarce Resources
- Horvitz, Suermondt, et al.
- 1989
(Show Context)
Citation Context ... algorithms for probabilistic inference include nestor [5], and, more recently, algorithms restricted to two-level (bipartite) noisy-OR belief networks [16, 28, 29], and other more general algorithms =-=[11, 17, 18, 30, 32, 34]-=-. Approximation algorithms are categorized by the nature of the bounds on the estimates that they produce and by the reliability with which the exact answer lies within these bounds. The following inf... |

89 |
Monte-Carlo approximation algorithms for enumeration problems
- Karp, Luby, et al.
- 1989
(Show Context)
Citation Context ... in an upper bound on N that guarantees that, for any ffl; ffi ? 0, Pr[(1 \Gamma ffl)sOEs(1 + ffl)] ? 1 \Gamma ffi; (1) withsequal to Pr[E = e] ors= Pr[X = x; E = e]. The Zero---One Estimator Theorem =-=[20]-=- gives an upper bound on the number N : N = 4 ffl 2 ln 2 ffi : Thus, provided that the probability Pr[X = x; E = e]sPr[E = e] is not too small--- for example, it is at least 1=n O(1) ---the number of ... |

67 |
Pseudorandom bits for constant depth circuits
- Nisan
- 1991
(Show Context)
Citation Context ...=1 Pr[W i = ! i j(W i )]j W=w;Z=z S / S + !=\Pi Output: \PiS=j\Phij Figure 2: The deterministic bounded-variance algorithm. 20 4.2.1 Boolean Circuits We discuss a model of computation for which Nisan =-=[24]-=- proves that we can stretch a short string of truly random bits into a long string of pseudorandom bits that appears random to this model. These models are constant-depth, unbounded fan-in Boolean cir... |

53 | S.M.: An optimal algorithm for Monte Carlo Estimation
- Dagum, Karp, et al.
- 2000
(Show Context)
Citation Context ...le. The bounded-variance algorithm is a simple variant of the known likelihoodweighting algorithm [13, 33], which employs recent results on the design of optimal algorithms for Monte Carlo simulation =-=[9]-=-. We consider an n-node belief network without extreme conditional probabilities and an evidence set E of constant size. We prove that, with a small failure probability ffi, the bounded-variance algor... |

46 |
Backward simulation in Bayesian networks
- Fung, Favero
- 1994
(Show Context)
Citation Context ...include straight simulation [25, 26], forward simulation, [15], likelihood weighting [13, 33], and randomized-approximation schemes [3, 4, 7, 8]. Variants of these methods such as backward simulation =-=[14]-=-, exist; Neal [23] provides a good overview of the theory of simulation-based algorithms. Search-based algorithms search the space of alternative instantiations to find the most probable instantiation... |

45 |
probabilistic causal model for diagnostic prblem solvingpart I : integrating symbolic causal inference with numeric probabili stic inference
- PENG, REGGIAJ
(Show Context)
Citation Context ...nds on the inference probabilities. Search-based algorithms for probabilistic inference include nestor [5], and, more recently, algorithms restricted to two-level (bipartite) noisy-OR belief networks =-=[16, 28, 29]-=-, and other more general algorithms [11, 17, 18, 30, 32, 34]. Approximation algorithms are categorized by the nature of the bounds on the estimates that they produce and by the reliability with which ... |

43 | Incremental probabilistic inference, in - D’Ambrosio - 1993 |

39 |
Search-based methods to bound diagnostic probabilities in very large belief nets
- Henrion
- 1991
(Show Context)
Citation Context ...nds on the inference probabilities. Search-based algorithms for probabilistic inference include nestor [5], and, more recently, algorithms restricted to two-level (bipartite) noisy-OR belief networks =-=[16, 28, 29]-=-, and other more general algorithms [11, 17, 18, 30, 32, 34]. Approximation algorithms are categorized by the nature of the bounds on the estimates that they produce and by the reliability with which ... |

37 |
A new algorithm for finding map assignments to belief networks
- Shimony, Charniak
- 1991
(Show Context)
Citation Context ... algorithms for probabilistic inference include nestor [5], and, more recently, algorithms restricted to two-level (bipartite) noisy-OR belief networks [16, 28, 29], and other more general algorithms =-=[11, 17, 18, 30, 32, 34]-=-. Approximation algorithms are categorized by the nature of the bounds on the estimates that they produce and by the reliability with which the exact answer lies within these bounds. The following inf... |

31 | Approximation of general independent distributions
- EVEN, GOLDEICH, et al.
- 1992
(Show Context)
Citation Context ...f deterministic-approximation algorithms for specific problems in RP that do not rely on unproved conjectures, such as the existence of pseudorandom generators, have also achieved subexponential time =-=[12, 22]-=-. Thus far, deterministicapproximation algorithms require substantially increased run time, in comparison to a randomized-approximation algorithm for the same problem. Deterministic algorithms, howeve... |

29 | Average-case analysis of a search algorithm for estimating prior and posterior probabilities in Bayesian networks with extreme probabilities
- Poole
- 1993
(Show Context)
Citation Context ... algorithms for probabilistic inference include nestor [5], and, more recently, algorithms restricted to two-level (bipartite) noisy-OR belief networks [16, 28, 29], and other more general algorithms =-=[11, 17, 18, 30, 32, 34]-=-. Approximation algorithms are categorized by the nature of the bounds on the estimates that they produce and by the reliability with which the exact answer lies within these bounds. The following inf... |

26 |
A randomized approximation algorithm for probabilistic inference on Bayesian belief networks
- Chavez, Cooper
- 1990
(Show Context)
Citation Context ...e random samples of the solution space. Simulation-based algorithms include straight simulation [25, 26], forward simulation, [15], likelihood weighting [13, 33], and randomized-approximation schemes =-=[3, 4, 7, 8]-=-. Variants of these methods such as backward simulation [14], exist; Neal [23] provides a good overview of the theory of simulation-based algorithms. Search-based algorithms search the space of altern... |

22 | Deterministic Approximate Counting of Depth-2 Circuits
- Luby, Velickovic, et al.
- 1993
(Show Context)
Citation Context ...f deterministic-approximation algorithms for specific problems in RP that do not rely on unproved conjectures, such as the existence of pseudorandom generators, have also achieved subexponential time =-=[12, 22]-=-. Thus far, deterministicapproximation algorithms require substantially increased run time, in comparison to a randomized-approximation algorithm for the same problem. Deterministic algorithms, howeve... |

17 | A Bayesian analysis of simulation algorithms for inference in belief networks
- Dagum, Horvitz
- 1993
(Show Context)
Citation Context ...e random samples of the solution space. Simulation-based algorithms include straight simulation [25, 26], forward simulation, [15], likelihood weighting [13, 33], and randomized-approximation schemes =-=[3, 4, 7, 8]-=-. Variants of these methods such as backward simulation [14], exist; Neal [23] provides a good overview of the theory of simulation-based algorithms. Search-based algorithms search the space of altern... |

9 | Optimal monte carlo estimation of belief network inference
- Pradhan, Dagum
- 1996
(Show Context)
Citation Context ...refore cannot run the bounded-variance algorithm to completion, suggest that the algorithm continues to provide reliable approximations, although we cannot guarantee the error in those approximations =-=[31]-=-. Although we may entertain the possibility that another design of a randomized algorithm might lead to polynomial solutions for inference probabilities regardless of the number of observed nodes, we ... |

8 |
A new algorithm for nding MAP assignments to belief networks
- Shimony, Charniak
- 1990
(Show Context)
Citation Context |

6 | Efficient SearchBased Inference for Noisy-OR Belief Networks: TopEpsilon
- Huang, Henrion
- 1997
(Show Context)
Citation Context |

4 |
An empirical evaluation of a randomized algorithm for probabilistic inference
- Chavez, Cooper
- 1989
(Show Context)
Citation Context |

4 |
Propagating uncertainty inBayesian networks by probabilistic logic sampling
- Henrion
- 1988
(Show Context)
Citation Context ...ations. Simulation-based algorithms use a source of random bits to generate random samples of the solution space. Simulation-based algorithms include straight simulation [25, 26], forward simulation, =-=[15]-=-, likelihood weighting [13, 33], and randomized-approximation schemes [3, 4, 7, 8]. Variants of these methods such as backward simulation [14], exist� Neal [23] provides a good overview of the theory ... |

2 |
nestor: A computer-based medical diganostic aid that integrates causal and probabilistic knowledge
- Cooper
- 1984
(Show Context)
Citation Context ...e instantiations to find the most probable instantiation. These methods yield upper and lower bounds on the inference probabilities. Search-based algorithms for probabilistic inference include nestor =-=[5]-=-, and, more recently, algorithms restricted to two-level (bipartite) noisy-OR belief networks [16, 28, 29], and other more general algorithms [11, 17, 18, 30, 32, 34]. Approximation algorithms are cat... |

1 |
Belief updating by enumeratiing high-probability independence-based assignments
- Santos, Shimony
- 1994
(Show Context)
Citation Context |