## Continuous Sigmoidal Belief Networks Trained Using Slice Sampling (0)

Venue: | Advances in Neural Information Processing Systems 9 |

Citations: | 9 - 2 self |

### BibTeX

@INPROCEEDINGS{Frey_continuoussigmoidal,

author = {Brendan Frey},

title = {Continuous Sigmoidal Belief Networks Trained Using Slice Sampling},

booktitle = {Advances in Neural Information Processing Systems 9},

year = {},

pages = {452--458},

publisher = {MIT Press}

}

### Years of Citing Articles

### OpenURL

### Abstract

Real-valued random hidden variables can be useful for modelling latent structure that explains correlations among observed variables. I propose a simple unit that adds zero-mean Gaussian noise to its input before passing it through a sigmoidal squashing function. Such units can produce a variety of useful behaviors, ranging from deterministic to binary stochastic to continuous stochastic. I show how "slice sampling" can be used for inference and learning in top-down networks of these units and demonstrate learning on two simple problems. 1 Introduction A variety of unsupervised connectionist models containing discrete-valued hidden units have been developed. These include Boltzmann machines (Hinton and Sejnowski 1986), binary sigmoidal belief networks (Neal 1992) and Helmholtz machines (Hinton et al. 1995; Dayan et al. 1995). However, some hidden variables, such as translation or scaling in images of shapes, are best represented using continuous values. Continuous-valued Bolt...

### Citations

7407 |
Probabilistic reasoning in intelligent systems: Networks of plausible inference
- Pearl
- 1988
(Show Context)
Citation Context ...these variables can adapt to be continuous or categorical. The proposed top-down model can be viewed as a continuous-valued belief network, which can be simulated by performing a quick top-down pass (=-=Pearl 1988-=-). Work done on continuous-valued belief networks has focussed mainly on Gaussian random variables that are linked linearly such that the joint distribution over all variables is also Gaussian (Pearl ... |

301 |
Learning and re-learning in boltzmann machines
- Hinton, Sejnowski
- 1986
(Show Context)
Citation Context ...monstrate learning on two simple problems. 1 Introduction A variety of unsupervised connectionist models containing discrete-valued hidden units have been developed. These include Boltzmann machines (=-=Hinton and Sejnowski 1986-=-), binary sigmoidal belief networks (Neal 1992) and Helmholtz machines (Hinton et al. 1995; Dayan et al. 1995). However, some hidden variables, such as translation or scaling in images of shapes, are ... |

232 | The wake-sleep algorithm for unsupervised neural networks
- Hinton, Dayan, et al.
- 1995
(Show Context)
Citation Context ... models containing discrete-valued hidden units have been developed. These include Boltzmann machines (Hinton and Sejnowski 1986), binary sigmoidal belief networks (Neal 1992) and Helmholtz machines (=-=Hinton et al. 1995-=-; Dayan et al. 1995). However, some hidden variables, such as translation or scaling in images of shapes, are best represented using continuous values. Continuous-valued Boltzmann machines have been d... |

198 | Varieties of Helmholtz machine
- Dayan, Hinton
- 1996
(Show Context)
Citation Context ...iscrete-valued hidden units have been developed. These include Boltzmann machines (Hinton and Sejnowski 1986), binary sigmoidal belief networks (Neal 1992) and Helmholtz machines (Hinton et al. 1995; =-=Dayan et al. 1995-=-). However, some hidden variables, such as translation or scaling in images of shapes, are best represented using continuous values. Continuous-valued Boltzmann machines have been developed (Movellan ... |

191 |
Connectionist learning of belief networks
- Neal
- 1991
(Show Context)
Citation Context ...riety of unsupervised connectionist models containing discrete-valued hidden units have been developed. These include Boltzmann machines (Hinton and Sejnowski 1986), binary sigmoidal belief networks (=-=Neal 1992-=-) and Helmholtz machines (Hinton et al. 1995; Dayan et al. 1995). However, some hidden variables, such as translation or scaling in images of shapes, are best represented using continuous values. Cont... |

147 | Independence properties of directed Markov fields - Lauritzen, Dawid, et al. - 1990 |

52 | Markov Chain Monte Carlo methods based on ŞslicingŤ the density function
- Neal
- 1997
(Show Context)
Citation Context ...mpling). However, it is difficult to sample from this distribution because it may have many peaks that range from broad to narrow. I use a new Markov chain Monte Carlo method called "slice sampli=-=ng" (Neal 1997-=-) to pick a new activity for each unit. Consider the problem of drawing a value y from a univariate distribution P (y) --- in this application, P (y) is the conditional distributionsp(y i jfy j g j 6=... |

44 | Learning Bayesian networks: A unification for discrete and Gaussian domains - Heckerman, Geiger - 1995 |

43 | Bayesian neural networks and density networks - MacKay - 1995 |

33 | Learning continuous probability distributions with symmetric diffusion networks - Movellan, McClelland - 1993 |

22 | Discovering structure in continuous variables using bayesian networks - Hofmann, Tresp - 1996 |

21 | Fast learning by bounding likelihoods in sigmoid type belief networks
- Jaakkola, Saul, et al.
- 1996
(Show Context)
Citation Context ...mma(x \Gammasi ) 2 =2oe 2 i ] p 2oe 2 i dx = Zsi \Gamma1 exp[\Gammax 2 =2oe 2 i ] p 2oe 2 i dx = \Phi isi oe i j : (5) This sort of stochastic activation is found in binary sigmoidal belief networks (=-=Jaakkola et al. 1996-=-) and in the decision-making components of mixture of expert models and hierarchical mixture of expert models. 3 Continuous sigmoidal belief networks If the mean of each unit depends on the activities... |

18 | EM optimization of latent-variable density models - Bishop, Svensén, et al. - 1996 |