Results 1  10
of
20
A New Evolutionary System for Evolving Artificial Neural Networks
 IEEE Transactions on Neural Networks
, 1996
"... This paper presents a new evolutionary system, i.e., EPNet, for evolving artificial neural networks (ANNs). The evolutionary algorithm used in EPNet is based on Fogel's evolutionary programming (EP) [1], [2], [3]. Unlike most previous studies on evolving ANNs, this paper puts its emphasis on evolvin ..."
Abstract

Cited by 155 (35 self)
 Add to MetaCart
This paper presents a new evolutionary system, i.e., EPNet, for evolving artificial neural networks (ANNs). The evolutionary algorithm used in EPNet is based on Fogel's evolutionary programming (EP) [1], [2], [3]. Unlike most previous studies on evolving ANNs, this paper puts its emphasis on evolving ANN's behaviours. This is one of the primary reasons why EP is adopted. Five mutation operators proposed in EPNet reflect such an emphasis on evolving behaviours. Close behavioural links between parents and their offspring are maintained by various mutations, such as partial training and node splitting. EPNet evolves ANN's architectures and connection weights (including biases 1 ) simultaneously in order to reduce the noise in fitness evaluation. The parsimony of evolved ANNs is encouraged by preferring node/connection deletion to addition. EPNet has been tested on a number of benchmark problems in machine learning and ANNs, such as the parity problem, the medical diagnosis problems (bre...
TimeSeries Forecasting Using Flexible Neural Tree Model
, 2004
"... Timeseries forecasting is an important research and application area. Much effort has been devoted over the past several decades to develop and improve the timeseries forecasting models. This paper introduces a new timeseries forecasting model based on the flexible neural tree (FNT). The FNT mode ..."
Abstract

Cited by 33 (15 self)
 Add to MetaCart
Timeseries forecasting is an important research and application area. Much effort has been devoted over the past several decades to develop and improve the timeseries forecasting models. This paper introduces a new timeseries forecasting model based on the flexible neural tree (FNT). The FNT model is generated initially as a flexible multilayer feedforward neural network and evolved using an evolutionary procedure. Very often it is a difficult task to select the proper input variables or timelags for constructing a timeseries model. Our research demonstrates that the FNT model is capable of handing the task automatically. The performance and effectiveness of the proposed method are evaluated using time series prediction problems and compared with those of related methods.
Generative Learning Structures and Processes for Generalized Connectionist Networks
, 1991
"... Massively parallel networks of relatively simple computing elements offer an attractive and versatile framework for exploring a variety of learning structures and processes for intelligent systems. This paper briefly summarizes the popular learning structures and processes used in such networks. It ..."
Abstract

Cited by 29 (18 self)
 Add to MetaCart
Massively parallel networks of relatively simple computing elements offer an attractive and versatile framework for exploring a variety of learning structures and processes for intelligent systems. This paper briefly summarizes the popular learning structures and processes used in such networks. It outlines a range of potentially more powerful alternatives for patterndirected inductive learning in such systems. It motivates and develops a class of new learning algorithms for massively parallel networks of simple computing elements. We call this class of learning processes generative for they offer a set of mechanisms for constructive and adaptive determination of the network architecture  the number of processing elements and the connectivity among them  as a function of experience. Generative learning algorithms attempt to overcome some of the limitations of some approaches to learning in networks that rely on modification of weights on the links within an otherwise fixed network t...
On Learning Simple Neural Concepts: From Halfspace Intersections to Neural Decision Lists
, 1992
"... In this paper, we take a close look at the problem of learning simple neural concepts under the uniform distribution of examples. By simple neural concepts we mean concepts that can be represented as simple combinations of perceptrons (halfspaces). One such class of concepts is the class of halfs ..."
Abstract

Cited by 27 (5 self)
 Add to MetaCart
In this paper, we take a close look at the problem of learning simple neural concepts under the uniform distribution of examples. By simple neural concepts we mean concepts that can be represented as simple combinations of perceptrons (halfspaces). One such class of concepts is the class of halfspace intersections. By formalizing the problem of learning halfspace intersections as a set covering problem, we are led to consider the following subproblem: given a set of non linearly separable examples, find the largest linearly separable subset of it. We give an approximation algorithm for this NPhard subproblem. Simulations, on both linearly and non linearly separable functions, show that this approximation algorithm works well under the uniform distribution, outperforming the Pocket algorithm used by many constructive neural algorithms. Based on this approximation algorithm, we present a greedy method for learning halfspace intersections. We also present extensive numerical...
A PopulationBased Learning Algorithm Which Learns Both Architectures and Weights of Neural Networks
 Chinese Journal of Advanced Software Research (Allerton
, 1996
"... One of the major issues in the field of artificial neural networks (ANNs) is the design of their architectures. There are strong biological and engineering evidences to support that the information processing capability of an ANN is determined by its architecture. This paper proposes a new populatio ..."
Abstract

Cited by 24 (13 self)
 Add to MetaCart
One of the major issues in the field of artificial neural networks (ANNs) is the design of their architectures. There are strong biological and engineering evidences to support that the information processing capability of an ANN is determined by its architecture. This paper proposes a new populationbased learning algorithm (PBLA) which learns both ANN's architecture and weights. The evolutionary approach is used to evolve a population of ANNs. Unlike other evolutionary approaches to ANN learning, each ANN (i.e., individual) in the population is evaluated by partial training rather than complete training. Substantial savings in computational cost can be achieved by such progressive partial training. This training process can change both ANN's architecture and weights. Our preliminary experiments have demonstrated the effectiveness of our algorithm. 1 Introduction One of the major issues in the field of ANNs is the design of their architectures. There are strong biological and enginee...
Neural Network Constructive Algorithms: Trading Generalization for Learning Efficiency?
, 1991
"... There are currently several types of constructive, or growth, algorithms available for training a feedforward neural network. This paper describes and explains the main ones, using a fundamental approach to the multilayer perceptron problemsolving mechanisms. The claimed convergence properties of ..."
Abstract

Cited by 11 (0 self)
 Add to MetaCart
There are currently several types of constructive, or growth, algorithms available for training a feedforward neural network. This paper describes and explains the main ones, using a fundamental approach to the multilayer perceptron problemsolving mechanisms. The claimed convergence properties of the algorithms are verified using just two mapping theorems, which consequently enables all the algorithms to be unified under a basic mechanism. The algorithms are compared and contrasted and the deficiencies of some highlighted. The fundamental reasons for the actual success of these algorithms are extracted, and used to suggest where they might most fruitfully be applied. A suspicion that they are not a panacea for all current neural network difficulties, and that one must somewhere along the line pay for the learning efficiency they promise, is developed into an argument that their generalization abilities will lie on average below that of backpropagation. Constructive Algorithms 1 1...
Symbol Processing Systems, Connectionist Networks, and Generalized Connectionist Networks
 IN S. GOONATILAKE AND S.KHEBBAL, EDITORS INTELLIGENT HYBRID SYSTEMS
, 1990
"... Many authors have suggested that SP (symbol processing) and CN (connectionist network) models offer radically, or even fundamentally, different paradigms for modeling intelligent behavior (see Schneider, 1987) and the design of intelligent systems. Others have argued that CN models have little to co ..."
Abstract

Cited by 8 (5 self)
 Add to MetaCart
Many authors have suggested that SP (symbol processing) and CN (connectionist network) models offer radically, or even fundamentally, different paradigms for modeling intelligent behavior (see Schneider, 1987) and the design of intelligent systems. Others have argued that CN models have little to contribute to our efforts to understand intelligence (Fodor & Pylyshyn, 1988). A critical examination of the popular characterizations of SP and CN models suggests that neither of these extreme positions is justified. There are many advantages to be gained by a synthesis of the best of both SP and CN approaches in the design of intelligent systems. The Generalized connectionist networks (GCN) (alternately called generalized neuromorphic systems (GNS)) introduced in this paper provide a framework for such a synthesis.
Optimization and Global Minimization Methods Suitable for Neural Networks
, 1998
"... Neural networks are usually trained using local, gradientbased procedures. Such methods frequently find suboptimal solutions being trapped in local minima. Optimization of neural structures and global minimization methods applied to network cost functions have strong influence on all aspects of n ..."
Abstract

Cited by 7 (4 self)
 Add to MetaCart
Neural networks are usually trained using local, gradientbased procedures. Such methods frequently find suboptimal solutions being trapped in local minima. Optimization of neural structures and global minimization methods applied to network cost functions have strong influence on all aspects of network performance. Recently genetic algorithms are frequently combined with neural methods to select best architectures and avoid drawbacks of local minimization methods. Many other global minimization methods are suitable for that purpose, although they are used rather rarely in this context. This paper provides a survey of such global methods, including some aspects of genetic algorithms.
Comparison of performance of variants of singlelayer perceptron algorithms on nonseparable data
 Neural, Parallel and Scientific Computation
, 2000
"... We present a detailed experimental comparison of the pocket algorithm, thermal perceptron, and barycentric correction procedure algorithms that most commonly used algorithms for training threshold logic units (TLUs). Each of these algorithms represent stable variants of the standard perceptron learn ..."
Abstract

Cited by 7 (1 self)
 Add to MetaCart
We present a detailed experimental comparison of the pocket algorithm, thermal perceptron, and barycentric correction procedure algorithms that most commonly used algorithms for training threshold logic units (TLUs). Each of these algorithms represent stable variants of the standard perceptron learning rule in that they guarantee convergence to zero classi cation errors on datasets that are linearly separable and attempt to classify as large a subset of the training patterns as possible for datasets that are not linearly separable. For datasets involving patterns distributed among M di erent categories (M>2) a group of M TLUs is trained, one for each of the output classes. These TLU's can be trained either independently or as a winnertakeall (WTA) group. The latter mechanism accounts for the interactions among the di erent output classes and exploits the fact that a pattern can ideally belong to only one of the M output classes. The extension of the pocket algorithm to the WTA output strategy is direct. In this paper we present heuristic extensions of the thermal perceptron and the barycentric correction procedure to WTA groups and empirically verify their performance. The performance of these algorithms was measured in a collection of carefully chosen benchmarks datasets. We report the training and generalization accuracies of these algorithms on the di erent datasets along with the learning time in seconds. In addition, a comparison of the learning speeds of the algorithms is indicated by means of learning curve plots on two datasets. We identify and report some distinguishing traits of these algorithms which could possibly enable making an informed choice of the training algorithm (combined with constructive learning algorithms) when certain characteristics of the dataset are known. 1
Constructive Training Methods for Feedforward Neural Networks with Binary Weights
, 1995
"... Quantization of the parameters of a Perceptron is a central problem in hardware implementation of neural networks using a numerical technology. A neural model with each weight limited to a small integer range will require little surface of silicon. Moreover, according to Ockham's razor principle, be ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
Quantization of the parameters of a Perceptron is a central problem in hardware implementation of neural networks using a numerical technology. A neural model with each weight limited to a small integer range will require little surface of silicon. Moreover, according to Ockham's razor principle, better generalization abilities can be expected from a simpler computational model. The price to pay for these benefits lies in the difficulty to train these kind of networks. This paper proposes essentially two new ideas for constructive training algorithms, and demonstrates their efficiency for the generation of feedforward networks composed of Boolean threshold gates with discrete weights. A proof of the convergence of these algorithms is given. Some numerical experiments have been carried out and the results are presented in terms of the size of the generated networks and of their generalization abilities. 1 Introduction Artificial neural networks (ANN) are proposed today as alternative...