## Computing Second Derivatives in Feed-Forward Networks: a Review (1994)

Venue: | IEEE Transactions on Neural Networks |

Citations: | 27 - 4 self |

### BibTeX

@ARTICLE{Buntine94computingsecond,

author = {Wray Buntine and Andreas S. Weigend},

title = {Computing Second Derivatives in Feed-Forward Networks: a Review},

journal = {IEEE Transactions on Neural Networks},

year = {1994},

volume = {5},

pages = {480--488}

}

### OpenURL

### Abstract

. The calculation of second derivatives is required by recent training and analyses techniques of connectionist networks, such as the elimination of superfluous weights, and the estimation of confidence intervals both for weights and network outputs. We here review and develop exact and approximate algorithms for calculating second derivatives. For networks with jwj weights, simply writing the full matrix of second derivatives requires O(jwj 2 ) operations. For networks of radial basis units or sigmoid units, exact calculation of the necessary intermediate terms requires of the order of 2h + 2 backward/forward-propagation passes where h is the number of hidden units in the network. We also review and compare three approximations (ignoring some components of the second derivative, numerical differentiation, and scoring). Our algorithms apply to arbitrary activation functions, networks, and error functions (for instance, with connections that skip layers, or radial basis functions, or ...