## Information Geometry of the EM and em Algorithms for Neural Networks (1995)

Venue: | Neural Networks |

Citations: | 101 - 8 self |

### BibTeX

@ARTICLE{Amari95informationgeometry,

author = {Shun-ichi Amari},

title = {Information Geometry of the EM and em Algorithms for Neural Networks},

journal = {Neural Networks},

year = {1995},

volume = {8},

pages = {1379--1408}

}

### Years of Citing Articles

### OpenURL

### Abstract

In order to realize an input-output relation given by noise-contaminated examples, it is effective to use a stochastic model of neural networks. A model network includes hidden units whose activation values are not specified nor observed. It is useful to estimate the hidden variables from the observed or specified input-output data based on the stochastic model. Two algorithms, the EM - and em-algorithms, have so far been proposed for this purpose. The EM-algorithm is an iterative statistical technique of using the conditional expectation, and the em-algorithm is a geometrical one given by information geometry. The em-algorithm minimizes iteratively the Kullback-Leibler divergence in the manifold of neural networks. These two algorithms are equivalent in most cases. The present paper gives a unified information geometrical framework for studying stochastic models of neural networks, by forcussing on the EM and em algorithms, and proves a condition which guarantees their equ...