## Adaptive Probabilistic Networks (1994)

Citations: | 10 - 2 self |

### BibTeX

@TECHREPORT{Russell94adaptiveprobabilistic,

author = {Stuart Russell and John Binder and Daphne Koller},

title = {Adaptive Probabilistic Networks},

institution = {},

year = {1994}

}

### OpenURL

### Abstract

Belief networks (or probabilistic networks) and neural networks are two forms of network representations that have been used in the development of intelligent systems in the field of artificial intelligence. Belief networks provide a concise representation of general probability distributions over a set of random variables, and facilitate exact calculation of the impact of evidence on propositions of interest. Neural networks, which represent parameterized algebraic combinations of nonlinear activation functions, have found widespread use as models of real neural systems and as function approximators because of their amenability to simple training algorithms. Furthermore, the simple, local nature of most neural network training algorithms provides a certain biological plausibility and allows for a massively parallel implementation. In this paper, we show that similar local learning algorithms can be derived for belief networks, and that these learning algorithms can operate using only ...

### Citations

8167 | Maximum likelihood from incomplete data via the EM algorithm
- Dempster, Laird, et al.
- 1977
(Show Context)
Citation Context ...ated work For a thorough introduction to belief networks, see [12]. The general problem of recovering distributions from data with missing values and hidden variables is addressed by the EM algorithm =-=[5]. Our algo-=-rithm can be seen as as a variant of EM in which the "maximize" phase is carried out by a gradient-following method. Lauritzen [9] also considers the application of EM to belief networks. Sp... |

7092 |
Probabilistic reasoning in intelligent systems: networks of plausible inference
- Pearl
- 1988
(Show Context)
Citation Context ...ent that must make decisions with incomplete information. Belief networks (also called causal networks and Bayesian networks) are currently the principal tool for representing probabilistic knowledge =-=[12]-=-. They provide a concise representation of general probability distributions over a set of propositional (or multi-valued) random variables. The basic task of a belief network is to calculate the prob... |

3584 | Optimization by Simulated Annealing
- Kirkpatrick, Gelatt, et al.
- 1983
(Show Context)
Citation Context ... after each hillclimbing phase reaches a maximum; and simulated annealing, in which some "downhill" steps are allowed stochastically, with a frequency depending on a decreasing "tempera=-=ture" schedule [8]-=-. In addition to the constraint that each CPT column must sum to 1, the search algorithm must also respect the constraint that CPT entries must fall in the range [0; 1]. It often occurs that the gradi... |

1293 |
Local Computations with Probabilities on Graphical Structures and their Application to Expert Systems
- Lauritzen, Spiegelhalter
- 1988
(Show Context)
Citation Context ...lculated in time linear in the size of the network. In general networks, the inference task is NP-hard [2], as is the corresponding approximation problem [4]. For general networks, a common technique =-=[10]-=- is to cluster variables in the network to form a join tree that is singly connected. Join tree algorithms calculate a posterior potential for clusters of nodes, where the entries in the potential tab... |

1081 | A Bayesian method for the induction of probabilistic networks from data
- Cooper, Herskovits
- 1992
(Show Context)
Citation Context ...y observable: In this case the problem is to reconstruct the topology of the network. An MAP analysis of the most likely network structure given the data has been carried out by Cooper and Herskovitz =-=[3]-=-, and by Heckerman et al. [7]. The resulting algorithms are capable of recovering fairly large networks from large data sets with a high degree of accuracy. However, because of the intractability of t... |

908 | Learning Bayesian networks: the combination of knowledge and statistical data
- Heckerman, Geiger, et al.
- 1995
(Show Context)
Citation Context ...e problem is to reconstruct the topology of the network. An MAP analysis of the most likely network structure given the data has been carried out by Cooper and Herskovitz [3], and by Heckerman et al. =-=[7]-=-. The resulting algorithms are capable of recovering fairly large networks from large data sets with a high degree of accuracy. However, because of the intractability of the problem of finding the bes... |

584 |
The computational complexity of probabilistic inference using bayesian belief networks
- Cooper
- 1990
(Show Context)
Citation Context ... any two nodes are connected by at most one undirected path, the posterior distribution can be calculated in time linear in the size of the network. In general networks, the inference task is NP-hard =-=[2]-=-, as is the corresponding approximation problem [4]. For general networks, a common technique [10] is to cluster variables in the network to form a join tree that is singly connected. Join tree algori... |

254 | Approximating probabilistic inference in Bayesian belief networks is NP-hard
- Dagum, Luby
- 1993
(Show Context)
Citation Context ...ted path, the posterior distribution can be calculated in time linear in the size of the network. In general networks, the inference task is NP-hard [2], as is the corresponding approximation problem =-=[4]-=-. For general networks, a common technique [10] is to cluster variables in the network to form a join tree that is singly connected. Join tree algorithms calculate a posterior potential for clusters o... |

216 |
The EM algorithm for graphical association models with missing data
- Lauritzen
- 1995
(Show Context)
Citation Context ...alues and hidden variables is addressed by the EM algorithm [5]. Our algorithm can be seen as as a variant of EM in which the "maximize" phase is carried out by a gradient-following method. =-=Lauritzen [9]-=- also considers the application of EM to belief networks. Spiegelhalter, Dawid, Lauritzen and Cowell [13] provide a thorough analysis of the statistical basis of belief network modification using Diri... |

190 |
Bayesian analysis in expert systems
- Spiegelhalter, Dawid, et al.
- 1993
(Show Context)
Citation Context ...nt of EM in which the "maximize" phase is carried out by a gradient-following method. Lauritzen [9] also considers the application of EM to belief networks. Spiegelhalter, Dawid, Lauritzen a=-=nd Cowell [13]-=- provide a thorough analysis of the statistical basis of belief network modification using Dirichlet priors, although they provide only a heuristic approximation for the hidden-variable case. Heckerma... |

183 |
Connectionist learning of belief networks
- Neal
- 1992
(Show Context)
Citation Context ...ic algorithm for recovering the structure of general networks in the fully observable case, building on the work of Cooper and Herskovits [3]. Gradient-following algorithms have been proposed by Neal =-=[11]-=-, who derives an expression for the likelihood gradient in sigmoid networks using stochastic simulation, and uses it to show that the Boltzmann Machine (a variety of neural network) is a special case ... |

136 |
Probabilistic similarity networks
- Heckerman
(Show Context)
Citation Context ...s for the remaining variables. Belief networks containing several thousand nodes and links have been used successfully to represent medical knowledge and to achieve high levels of diagnostic accuracy =-=[6]-=-, among other tasks. (a) (b) C C ~C ~A~B A B A ~B ~A B 0.3 0.7 0.9 0.1 0.5 0.5 0.1 0.9 Burglary Earthquake Alarm NeighbourCalls RadioReport Figure 1: (a) A belief network node with associated conditio... |

120 | Fundamental concepts of qualitative probabilistic networks - Wellman - 1990 |

1 |
Probabilistic resolution of anaphoric inference
- Burger, Connolly
- 1994
(Show Context)
Citation Context ...lts show that an extremely close connection exists between neural and belief networks. Our results can be seen as a generalization of Neal's, and yield a more efficient algorithm. Burger and Connolly =-=[1]-=- also apply neural network techniques to learning belief networks, using a somewhat ad hoc error function to derive an error gradient for polytree networks. 2 Learning networks with fixed structure Ex... |