Results 1 
5 of
5
Mean Field Theory for Sigmoid Belief Networks
 Journal of Artificial Intelligence Research
, 1996
"... We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics. ..."
Abstract

Cited by 116 (12 self)
 Add to MetaCart
We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics.
A Mean Field Learning Algorithm For Unsupervised Neural Networks
, 1999
"... . We introduce a learning algorithm for unsupervised neural networks based on ideas from statistical mechanics. The algorithm is derived from a mean field approximation for large, layered sigmoid belief networks. We show how to (approximately) infer the statistics of these networks without resort to ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
. We introduce a learning algorithm for unsupervised neural networks based on ideas from statistical mechanics. The algorithm is derived from a mean field approximation for large, layered sigmoid belief networks. We show how to (approximately) infer the statistics of these networks without resort to sampling. This is done by solving the mean field equations, which relate the statistics of each unit to those of its Markov blanket. Using these statistics as target values, the weights in the network are adapted by a local delta rule. We evaluate the strengths and weaknesses of these networks for problems in statistical pattern recognition. 1. Introduction Multilayer neural networks trained by backpropagation provide a versatile framework for statistical pattern recognition. They are popular for many reasons, including the simplicity of the learning rule and the potential for discovering hidden, distributed representations of the problem space. Nevertheless, there are many issues that are...
Attractor Dynamics in Feedforward Neural Networks
"... this article, we show that this linkage of attractor dynamics and probabilistic inference is not limited to symmetric networks or (equivalently) to models represented as undirected graphs. We investigate an attractor dynamics for feedforward networks, or directed acyclic graphs (DAGs); these are net ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
this article, we show that this linkage of attractor dynamics and probabilistic inference is not limited to symmetric networks or (equivalently) to models represented as undirected graphs. We investigate an attractor dynamics for feedforward networks, or directed acyclic graphs (DAGs); these are networks with directed edges but no directed loops. The probabilistic models represented by DAGs are known as Bayesian networks, and together with MRFs, they comprise the class of probabilistic models known as graphical models (Lauritzen, 1996). Like their undirected counterparts, Bayesian networks have been proposed as models of both artificial and biological intelligence (Pearl, 1988).
An Introduction to Variational Methods for Graphical Methods
 Machine Learning
, 1998
"... . This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields). We present a number of examples of graphical models, including the QMRDT database, the sigmoid belief network, the Boltzmann m ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
. This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields). We present a number of examples of graphical models, including the QMRDT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, which exploit laws of large numbers to transform the original graphical model into a simplified graphical model in which inference is efficient. Inference in the simpified model provides bounds on probabilities of interest in the original model. We describe a general framework for generating variational transformations based on convex duality. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.
Prior Information and Generalized Questions
, 1996
"... In learning problems available information is usually divided into two categories: examples of function values (or training data) and prior information (e.g. a smoothness constraint). ..."
Abstract

Cited by 7 (4 self)
 Add to MetaCart
In learning problems available information is usually divided into two categories: examples of function values (or training data) and prior information (e.g. a smoothness constraint).