Standard methods for inducing both the structure and weight values of recurrent neural networks fit an assumed class of architectures to every task. This simplification is necessary because the interactions between network structure and function are not well understood. Evolutionary computation, which includes genetic algorithms and evolutionary programming, is a population-based search method that has shown promise in such complex tasks. This paper argues that genetic algorithms are inappropriate for network acquisition and describes an evolutionary program, called GNARL, that simultaneously acquires both the structure and weights for recurrent networks. This algorithm's empirical acquisition method allows for the emergence of complex behaviors and topologies that are potentially excluded by the artificial architectural constraints imposed in standard network induction methods. To Appear in: IEEE Transactions on Neural Networks January The Ohio State University January 17, 1996 1 ...

Evolutionary computation has started to receive significant attention during the last decade, although the origins can be traced back to the late 1950s. This article surveys the history as well as the current state of this rapidly growing field. We describe the purpose, the general structure and the working principles of different approaches, including genetic algorithms (GA) (with links to genetic programming (GP) and classifier systems (CS)), evolution strategies (ES), and evolutionary programming (EP), by analysis and comparison of their most important constituents (i.e., representations, variation operators, reproduction and selection mechanism). Finally, we give a brief overview on the manifold of application domains, although this necessarily must remain incomplete.

Evolutionary programming (EP) has been applied with success to many numerical and combinatorial optimization problems in recent years. EP has rather slow convergence rates, however, on some function optimization problems. In this paper, a "fast EP" (FEP) is proposed which uses a Cauchy instead of Gaussian mutation as the primary search operator. The relationship between FEP and classical EP (CEP) is similar to that between fast simulated annealing and the classical version. Both analytical and empirical studies have been carried out to evaluate the performance of FEP and CEP for different function optimization problems. This paper shows that FEP is very good at search in a large neighborhood while CEP is better at search in a small local neighborhood. For a suite of 23 benchmark problems, FEP performs much better than CEP for multimodal functions with many local minima while being comparable to CEP in performance for unimodal and multimodal functions with only a few local minima. This paper also shows the relationship between the search step size and the probability of finding a global optimum and thus explains why FEP performs better than CEP on some functions but not on others. In addition, the importance of the neighborhood size and its relationship to the probability of finding a near-optimum is investigated. Based on these analyses, an improved FEP (IFEP) is proposed and tested empirically. This technique mixes different search operators (mutations). The experimental results show that IFEP performs better than or as well as the better of FEP and CEP for most benchmark problems tested.

This paper analyzes the convergence properties of the canonical genetic algorithm (CGA) with mutation, crossover and proportional reproduction applied to static optimization problems. It is proved by means of homogeneous finite Markov chain analysis that a CGA will never converge to the global optimum regardless of the initialization, crossover operator and objective function. But variants of CGAs that always maintain the best solution in the population, either before or after selection, are shown to converge to the global optimum due to the irreducibility property of the underlying original nonconvergent CGA. These results are discussed with respect to the schema theorem. Keywords: canonical genetic algorithm, global convergence, Markov chains, schema theorem 1 Introduction Canonical genetic algorithms (CGA) as introduced in [1] are often used to tackle static optimization problems of the type maxff(b) j b 2 IB l g (1) assuming that 0 ! f(b) ! 1 for all b 2 IB l = f0; 1g l and ...

Due to its independence of the actual search space and its impact on the exploration-exploitation tradeoff, selection is an important operator in any kind of Evolutionary Algorithm. In this paper, all important selection operators are discussed and quantitatively compared with respect to their selective pressure. The comparison clarifies that only a few really different and useful selection operators exist: Proportional selection (in combination with a scaling method), linear ranking, tournament selection, and (¯,)-selection (respectively (¯+)-selection). Their selective pressure increases in the order as they are listed here. The theoretical results are confirmed by an experimental investigation using a Genetic Algorithm with different selection methods on a simple unimodal objective function. I. Introduction Evolutionary Algorithms (EAs) are a class of direct probabilistic search algorithms based on the model of organic evolution. Currently, Genetic Algorithms (GAs) [17; 12], Evolu...

This paper analyzes the Self-Adaptation (SA) algorithm widely used to adapt strategy parameters of the Evolution Strategy (ES) in order to obtain maximal ES-performance. The investigations are concentrated on the adaptation of one general mutation strength oe (called oeSA) in (1; ) ESs. The hypersphere serves as the fitness model. Starting from an introduction into the basic concept of self-adaptation, a framework for the analysis of oeSA is developed on two levels: a microscopic level concerning the description of the stochastic changes from one generation to the next, and a macroscopic level describing the evolutionary dynamics of the oeSA over the time (generations). The oe-SA requires the fixing of a new strategy parameter, the so-called learning parameter. The influence of this parameter on the ES performance is investigated and rules for its tuning are presented and discussed. The results of the theoretical analysis are compared with ES experiments and it will be shown that apply...

Evolutionary programming is a method for simulating evolution that has been investigated for almost 40 years. When originally introduced, the available computing equipment was quite slow and difficult to use as measured by current standards. This paper provides a series of experiments that follow the framework of the original approach from the early 1960s, brought up to date with current computing machinery. A brief review of evolutionary programming and its relationship to other methods of evolutionary computation, specifically genetic algorithms and evolution strategies, is also offered. Keywords: evolutionary programming, evolutionary computation, forecasting, control. 1. INTRODUCTION There are three main lines of investigation within the current framework of evolutionary computation: (1) genetic algorithms, (2) evolution strategies, and (3) evolutionary programming. Reviews of these methods are offered in several recent books 1-5 . Each of these methods has developed over more ...

Genetic algorithms are computational models of evolution that play a central role in many artificial-life models. We review the history and current scope of research on genetic algorithms in artificial life, using illustrative examples in which the genetic algorithm is used to study how learning and evolution interact, and to model ecosystems, immune system, cognitive systems, and social systems. We also outline a number of open questions and future directions for genetic algorithms in artificial-life research.

This paper offers sufficient conditions to prove global convergence of non--elitist evolutionary algorithms. If these conditions can be applied they yield bounds of the convergence rate as a by--product. This is demonstrated by an example that can be calculated exactly. KeyW ords--- global convergence, non--elitist evolutionary algorithm, martingale theory I. Introduction Evolutionary algorithms (EAs) represent a class of stochastic optimization algorithms in which principles of organic evolution are regarded as rules for optimization. They are often applied to parameter optimization problems [1] when specialized techniques are not available or standard methods fail to give satisfactory answers due to multimodality, nondifferentiability or discontinuities of the problem under consideration. In general, evolutionary algorithms may be classified as elitist or non--elitist strategies. The characteristic feature of elitist strategies is that they always maintain the best solution (indiv...

Evolutionary programming (EP) has been applied to many numerical and combinatorial optimisation problems successfully in recent years. One disadvantage of EP is its slow convergence to a good near optimum for some function optimisation problems. In this paper, we propose a fast EP (FEP) which uses a Cauchy instead of Gaussian mutation operator as the primary search operator. The relationship between FEP and classical EP (CEP) is similar to that between the fast simulated annealing and the classical version. Extensive empirical studies have been carried out to evaluate the performance of FEP for different function optimisation problems. Fifty runs have been conducted for each of the 23 test functions in our studies. Our experimental results show that FEP performs much better than CEP for multi-modal functions with many local minima while being comparable to CEP in performance for unimodal and multi-modal functions with only a few local minima. We emphasise in the paper that no single al...