Neural networks and genetic algorithms are two relatively young research areas that were subject to a steadily growing interest during the past years. Both models are inspired by nature, but whereas neural networks are concerned with learning of an individual (phenotypic learning), evolutionary algorithms deal with a population's adaptation to a changing environment (genotypic learning). This paper focuses on the intersection of neural networks and evolutionary computation, namely on how evolutionary algorithms can be used to assist neural network design and training. The purpose of the paper is to set forth the general considerations that have to be made when designing an algorithm in this area and to give an overview on how researchers addressed these issues in the past.

Clustering algorithms aim at modelling fuzzy (i.e., ambiguous) unlabeled patterns efficiently. Our goal is to propose a theoretical framework where clustering systems can be compared on the basis of their learning strategies. In the first part of this work, the following issues are reviewed: relative (probabilistic) and absolute (possibilistic) fuzzy membership functions and their relationships to the Bayes rule, batch and on-line learning, growing and pruning networks, modular network architectures, topologically perfect mapping, ecological nets and neuro-fuzziness. From this discussion an equivalence between the concepts of fuzzy clustering and soft competitive learning in clustering algorithms is proposed as a unifying framework in the comparison of clustering systems. Moreover, a set of functional attributes is selected for use as dictionary entries in our comparison. In the second part of this paper, five clustering algorithms taken from the literature are reviewed and compared on...

This paper describes how through simple means a genetic search towards optimal neural network architectures can be improved, both in the convergence speed as in the quality of the final result. This result can be theoretically explained with the Baldwin effect, which is implemented here not just by the learning process of the network alone, but also by changing the network architecture as part of the learning procedure. This can be seen as a combination of two different techniques, both helping and improving on simple genetic search.

To embed modularity (i.e. to perform a local and encapsulated computation) into neural networks (NN) leads to many advantages. Hence, the development of a general model of modular neural networks (MNN) will enable a broader use of Neural Networks (NN). However, some important issues remain to be solved to enable a systematic use of MNN. In a practical point of view, the most important matter concerns the decomposition of the task into subtasks. We have introduced here the concept of vertical and horizontal decomposition in order to classify the existing modular models capable of performing a selfdecomposition. The modular models available for a horizontal self-decomposition (i.e. a clustering of the input space) are mainly the Local Model Network (LMN) and the algorithm of Jacobs and Jordan. Those two algorithms appear complementary. The convergence of the latter one is not ensured but the criterion it uses for decomposing the input space is far more ambitious and efficient than the s...

The use of "global neural networks" (as the back propagation neural network) and "clustering neural networks" (as the radial basis function neural network) leads each other to different advantages and inconvenients. The combination of the desirable features ot those two neural ways of computation is achieved by the use of Modular Neural Networks (MNN). In addition, a considerable advantage can emerge from the use of such a MNN: an interpreatable and relevant neural representation about the plant's behaviour. This very desirable feature for function approximation and especially for control problems, is what lake other neural models. This feature is so important that we introduce it as a way to differenciate MNN between other local computation models. However, to enable a systematic use of MNN three steps have to be achieved. First of all, the task has to be decomposed into subtasks, then the neural modules have to be properly organised considering the subtasks and finally a way of commu...

Learning a large number of simple local concepts is both faster and easier than learning a single global concept. Inspired by this principle of divide and conquer, a number of modular learning approaches have been proposed by the computational intelligence community. In modular learning, the classification/regression/clustering problem is first decomposed into a number of simpler subproblems, a module is learned for each of these subproblems, and finally their results are integrated by a suitable combining method. Mixtures of experts and clustering are two of the techniques that are describable in this paradigm. In this paper we present a broad framework for Generalized Associative Modular Learning Systems (GAMLS). Modularity is introduced through soft association of each training pattern with every module. The coupled problems of learning the module parameters and learning associations are solved iteratively using deterministic annealing. Starting at a high temperature with only one modu...

Abstract not found

this paper, we give a broad overview of our research into attribute grammar representations, from the basic and known capabilities, to the current ideas being addressed, to the future directions of our research.

The development and theoretical analysis of neural network architectures may be improved with the availability of techniques which allow the systematic representation and generation of classes of architectures. Recent work on the genetic optimization of neural networks has led to new ideas on how to encode neural network architectures abstractly as grammars. Extending this approach, we have devised an encoding system that uses an attribute grammar in which the evaluation of both synthesized and inherited attributes within a generated parse tree provides the details of the connectivity of the network. Comparison with cellular encoding and the geometry-oriented variation of cellular encoding suggests that attribute grammar encoding is simpler, easier to use, and has more potential as a technique for effectively generating neural networks. 1. Introduction Existing neural network architectures vary greatly in their structural form, learning styles, and functional characteristics. Compara...

This paper describes the SCAN (Signal Channelling Attentional Network) model, a scalable neural-network model for attentional scanning. The building block of SCAN is a gating lattice, a sparsely-connected neural network defined as a special case of the Ising lattice from statistical mechanics. The process of spatial selection through covert attention is interpreted as a biological solution to the problem of translation-invariant pattern processing. In SCAN, a sequence of pattern translations combines active selection with translation-invariant processing. Selected patterns are channelled through a gating network, formed by a hierarchical fractal structure of gating lattices, and mapped onto an output window. We show how the incorporation of an expectation-generating classifier network (e.g., Carpenter and Grossberg's ART network) into SCAN allows attentional selection to be driven by expectation. Simulation studies show the SCAN model to be capable of attending and identifying object p...