The Helmholtz Machine

The Helmholtz Machine (1995)

Cached

Download Links

by Peter Dayan , Geoffrey E. Hinton , Radford M. Neal , Richard S. Zemel

Citations:

BibTeX

@MISC{Dayan95thehelmholtz,
    author = {Peter Dayan and Geoffrey E. Hinton and Radford M. Neal and Richard S. Zemel},
    title = { The Helmholtz Machine},
    year = {1995}
}

Years of Citing Articles

Bookmark

OpenURL

Abstract

Discovering the structure inherent in a set of patterns is a fundamental aim of statistical inference or learning. One fruitful approach is to build a parameterized stochastic generative model, independent draws from which are likely to produce the patterns. For all but the simplest generative models, each pattern can be generated in exponentially many ways. It is thus intractable to adjust the parameters to maximize the probability of the observed patterns. We describe a way of finessing this combinatorial explosion by maximizing an easily computed lower bound on the probability of the observations. Our method can be viewed as a form of hierarchical self-supervised learning that may relate to the function of bottom-up and top-down cortical processing pathways.

Citations

6236	Maximum likelihood from incomplete data via EM algorithm - Dempster, Laird, et al. - 1977
5662	Probabilistic reasoning in intelligent systems: networks of plausible inference - Pearl - 1988
1298	Neural networks and physical systems with emergent collective computational abilities - Hopfield - 1982
1060	Learning to predict by the methods of temporal differences - Sutton - 1988
936	Self-Organized Formation of Topologically Correct Feature Maps - Kohonen - 1982
872	Information Theory and Statistics - Kullback - 1959
721	The organization of behavior - Hebb - 1949
612	A view of the EM algorithm that justifies incremental, sparse, and other variants - Neal, Hinton - 1998
575	A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains - LE, Petrie, et al. - 1970
448	Probabilistic inference using Markov chain Monte Carlo methods - Neal - 1993
427	Integrated architectures for learning, planning, and reacting based on approximating integrated architectures for learning, planning, and reacting based on approximating dynamic programming - Sutton - 1990
375	Factorial hidden markov models - Ghahramani, Jordan - 1997
368	A Learning Algorithm for Continually Running Fully Recurrent Neural Networks’, Neural computation - Williams, Zipser - 1989
364	A learning algorithm for Boltzmann machines - Ackley, Hinton - 1985
359	A massively parallel architecture for a self-organizing neural pattern recognition machine - Carpenter, Grossberg - 1987
353	Training products of experts by minimizing contrastive divergence - Hinton
262	Simple statistical gradient-following algorithms for connectionist reinforcement learning - Williams - 1992
247	Forward models: Supervised learning with a distal teacher - Jordan, Rumelhart - 1992
245	Learning and relearning in Boltzmann machines - Hinton, Sejnowski - 1986
241	A Fast Learning Algorithm for Deep Belief Nets - Hinton, Osindero, et al.
197	G: The EM algorithm for mixtures of factor analyzers - Ghahramani, Hinton - 1997
191	The ‘‘wake-sleep’’ algorithm for unsupervised neural networks - GE, Dayan, et al. - 1995
191	Information processing in dynamical systems: Foundations of harmony theory - Smolensky - 1986
187	Unsupervised learning - Barlow - 1989
185	Learning and sequential decision making - Barto, Sutton - 1990
173	An inequality with applications to statistical estimation for probabilistic functions of a Markov process and to models for ecology - Baum, Eagon - 1967
159	Connectionist learning of belief networks - Neal - 1992
157	Supervised learning from incomplete data via an EM approach - Ghahramani, Jordan - 1994
153	Adaptative Signal Processing - Widrow, Sterns - 1985
130	An emergent model of orientation selectivity in cat visual cortical simple cells - Somers, Nelson, et al. - 1995
127	A mean field theory learning algorithm for neural networks - Peterson, Anderson - 1987
116	Self-organizing neural network that discovers surfaces in random-dot stereograms. Nature 355 - Becker, GE - 1992
114	Theory of orientation tuning in visual cortex - Ben-Yishai, Bar-Or, et al. - 1995
114	On convergence properties of the EM algorithm for Gaussian mixtures - Xu, Jordan - 1996
113	An Introduction to Latent Variable Models - Everitt - 1984
113	Keeping neural networks simple by minimizing the description length of weights - Hinton, Camp - 1993
111	Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1: 1–47 - DJ, DC - 1991
102	Mean Field Theory for Sigmoid Belief Networks - Saul, Jaakkola, et al. - 1996
98	Autoencoders, minimum description length, and Helmholtz free energy - Hinton, Zemel - 1994
96	Learning internal representation by back-propagating errors - Rumelhart, Hinton, et al. - 1986
89	Convergence results for the em approach to mixtures of experts architectures - Jordan, Xu - 1995
81	Backpropagation: The basic theory - Rumelhart, Durbin, et al. - 1995
74	EM algorithms for ML factor analysis - Rubin, Thayer - 1982
74	Restricted Boltzmann machines for collaborative filtering - Salakhutdinov, Mnih, et al. - 2007
65	Neuronal architectures for pattern-theoretic problems - Mumford - 1994
61	Pattern recognizing stochastic learning automata - Barto, Anandan - 1985
60	A multiple cause mixture model for unsupervised learning - Saund - 1995
58	Training restricted Boltzmann machines using approximations to the likelihood gradient - Tieleman - 2008
56	Deterministic Boltzmann learning performs steepest descent in weight-space - Hinton - 1990
54	On contrastive divergence learning - Carreira-Perpiñán, Hinton - 2005