Abstract

In this paper, we show that using MSE-optimal linear combinations of a set of trained feedforward networks may significantly improve the accuracy of approximating a function and its first and second order derivatives. Our results are compared to the accuracies achieved by the single best network and by the simple averaging of the outputs of the trained networks. 1 Introduction Feedforward neural networks (FNN) are widely used for function approximation. They are considered universal approximators capable of approximating an unknown mapping and its derivatives arbitrarily well (Hornik et al. 1990). Approximating the derivatives, that is the derivatives of the output with respect to the inputs, is of significant importance in many applications. For example in process optimization, the first and second order derivatives obtained from a neural network, which was trained on the process response, may be used in approximating the gradient vector and the Hessian matrix of the process response...

Citations