Computing Second Derivatives in Feed-Forward Networks: a Review

Computing Second Derivatives in Feed-Forward Networks: a Review (1994)

Cached

Download Links

[archive.cis.ohio-state.edu]

by Wray Buntine , Andreas S. Weigend

Venue:	IEEE Transactions on Neural Networks
Citations:	22 - 4 self

Versions

Version History

Metadata Version 1

Datum	Value	Source
TITLE	Computing second derivatives in feed-forward networks: A review	INFERENCE
AUTHOR NAME	Wray Buntine	user correction - Legacy Corrections
AUTHOR AFFIL	RIACS & NASA Ames Research Center	user correction - Legacy Corrections
AUTHOR ADDR	Mail Stop 269--2; Moffet Field, CA 94035--1000, USA	user correction - Legacy Corrections
AUTHOR NAME	Andreas S. Weigend	user correction - Legacy Corrections
AUTHOR AFFIL	; Xerox PARC; Department of Computer Science and Institute of Cognitive Science,	user correction - Legacy Corrections
AUTHOR ADDR	; 3333 Coyote Hill Road; Palo Alto, CA 94304, USA; University of Colorado, Boulder, CO 80309-0430, USA.	user correction - Legacy Corrections
ABSTRACT	. The calculation of second derivatives is required by recent training and analyses techniques of connectionist networks, such as the elimination of superfluous weights, and the estimation of confidence intervals both for weights and network outputs. We here review and develop exact and approximate algorithms for calculating second derivatives. For networks with jwj weights, simply writing the full matrix of second derivatives requires O(jwj 2 ) operations. For networks of radial basis units or sigmoid units, exact calculation of the necessary intermediate terms requires of the order of 2h + 2 backward/forward-propagation passes where h is the number of hidden units in the network. We also review and compare three approximations (ignoring some components of the second derivative, numerical differentiation, and scoring). Our algorithms apply to arbitrary activation functions, networks, and error functions (for instance, with connections that skip layers, or radial basis functions, or ...	user correction - Legacy Corrections
YEAR	1994	INFERENCE
VENUE	IEEE Transactions on Neural Networks	INFERENCE
VENUE TYPE	JOURNAL	INFERENCE
PAGES	480--488	INFERENCE
VOLUME	5	INFERENCE
CITATIONS	26 found	ParsCit 1.0