This paper presents a new evolutionary system, i.e., EPNet, for evolving artificial neural networks (ANNs). The evolutionary algorithm used in EPNet is based on Fogel's evolutionary programming (EP) [1], [2], [3]. Unlike most previous studies on evolving ANNs, this paper puts its emphasis on evolving ANN's behaviours. This is one of the primary reasons why EP is adopted. Five mutation operators proposed in EPNet reflect such an emphasis on evolving behaviours. Close behavioural links between parents and their offspring are maintained by various mutations, such as partial training and node splitting. EPNet evolves ANN's architectures and connection weights (including biases 1 ) simultaneously in order to reduce the noise in fitness evaluation. The parsimony of evolved ANNs is encouraged by preferring node/connection deletion to addition. EPNet has been tested on a number of benchmark problems in machine learning and ANNs, such as the parity problem, the medical diagnosis problems (bre...

The Rprop algorithm proposed by Riedmiller and Braun is one of the best performing first-order learning methods for neural networks. We introduce modifications of the algorithm that improve its learning speed. The resulting speedup is experimentally shown for a set of neural network learning tasks as well as for artificial error surfaces.

The intuitive expectation is that the scheme used to encode the neural network in the chromosome should be critical to the success of evolving neural networks to solve difficult problems. In 1990 Kitano [1] published an encoding scheme based on contextfree parallel matrix rewriting. The method allowed compact, finite, chromosomes to grow neural networks of potentially infinite size. Results were presented that demonstrated superior evolutionary properties of the matrix rewriting method compared to a simple direct encoding. In this paper, we present results that contradict those findings, and demonstrate that a genetic algorithm (GA) using a direct encoding can find good individuals just as efficiently as a GA using matrix rewriting. I. Introduction The intuitive expectation is that the scheme used to encode the neural network in the chromosome should be critical to the success of evolving neural networks to solve difficult problems. In 1990 Kitano [1] published an encoding scheme base...

Recurrent neural networks are particularly useful for processing time sequences and simulating dynamical systems. However, methods for building recurrent architectures have been hindered by the fact that available training algorithms are considerably more complex than those for feedforward networks. In this paper

. This paper describes the forward-backward module: a simple building block that allows the evolution of neural networks with intrinsic supervised learning ability. This expands the range of networks that can be efficiently evolved compared to previous approaches, and also enables the networks to be invertible i.e. once a network has been evolved for a given problem domain, and trained on a particular dataset, the network can then be run backwards to observe what kind of mapping has been learned, or for use in control problems. A demonstration is given of the kind of self-training networks that could be evolved. 1 Introduction Despite much research into evolving neural networks, there is relatively little work published on evolving learning behaviours for neural networks. Within this area we can identify four kinds of approach: -- Write down a set of learning rules with variable parameters for a given network architecture, then evolve the choice of rules and their parameters. -- Evo...

In this paper a new approach to the evolution of neural networks is presented. A linear chromosome combined with a grid-based representation of the network, and a new crossover operator, allow the evolution of the architecture and the weights simultaneously. In our approach there is no need for a separate weight optimization procedure and networks with more than one type of activation function can be evolved. A pruning strategy is also introduced, which leads to the generation of solutions with varying degrees of complexity. lesults of the application of the method to several binary classification problems are reported.