This paper looks at the use of small populations in Genetic Programming (GP), where the trend in the literature appears to be towards using as large a population as possible, which requires more memory resources and CPU-usage is less efficient. Dynamic Subset Selection (DSS) and Limited Error Fitness (LEF) are two different, adaptive variations of the standard supervised learning method used in GP. This paper compares the performance of GP, GP+DSS, and GP+LEF, on a 958 case classification problem, using a small population size of 50. A similar comparison between GP and GP+DSS is done on a larger and messier 3772 case classification problem. For both problems, GP+DSS with the small population size consistently produces a better answer using fewer tree evaluations than other runs using much larger populations. Even standard GP can be seen to perform well with the much smaller population size, indicating that it is certainly worth an exploratory run or three with a small population size b...

Recent interest in human-level intelligence suggests a rethink of the role of machine learning in computational intelligence. We argue that without cognitive learning the goal of achieving human-level synthetic intelligence is far from completion. Here we review the principles underlying human learning and memory, and identify three of them, i.e., continuity, glocality, and compositionality, as the most fundamental to human-level machine learning. We then propose the recently-developed hypernetwork model as a candidate architecture for cognitive learning and memory. Hypernetworks are a random hypergraph structure higher-order probabilistic relations of data by an evolutionary self-organizing process based on molecular selfassembly. The chemically-based massive interaction for information organization and processing in the molecular hypernetworks, referred to as hyperinteractionism, is contrasted with the symbolist, connectionist, and dynamicist approaches to mind and intelligence. We demonstrate the generative learning capability of the hypernetworks to simulate linguistic recall memory, visual imagery, and language-vision crossmodal translation based on a video corpus of movies and dramas in a multimodal memory game environment. We also offer prospects for the hyperinteractionistic molecular mind approach to a unified theory of cognitive learning.

A new incremental learning algorithm for function approximation problems is presented where the neural network learner dynamically selects during training the most informative patterns from a candidate training set. The incremental learning algorithm uses its current knowledge about the function to be approximated, in the form of output sensitivity information, to incrementally grow the training set with patterns that have the highest influence on the learning objective.

Genetic programming has been successfully applied to evolve computer programs for solving a variety of interesting problems. In the previous work we introduced the breeder genetic programming (BGP) method that has Occam's razor in its fitness measure to evolve minimal size multilayer perceptrons. In this paper we apply the method to synthesis of sigma-pi neural networks. Unlike perceptron architectures, sigma-pi networks use product units as well as summation units to build higher-order terms. The effectiveness of the method is demonstrated on benchmark problems. Simulation results on noisy data suggest that BGP not only improves the generalization performance, it can also accelerate the convergence speed. I. Introduction Genetic programming has been successfully used to evolve computer programs for solving many interesting problems in artificial intelligence and artificial life [4, 16, 18, 19]. Similar to usual genetic algorithms (GAs), genetic programming (GP) starts with a populatio...

this paper we present Evolutionary TDSs, a GA--based active selection method, where the patterns of the (sub)--optimal TDSs are selected in parallel

Neural networks are usually trained on all available data. Support Vector Machines start from all data but near the end of the training use only a small subset of vectors near the decision border. The same learning strategy may be used in neural networks, independently of the actual optimization method used. Feedforward step is used to identify vectors that will not contribute to optimization. Threshold for acceptance of useful vectors for training is dynamically adjusted during learning to avoid excessive oscillations in the number of support vectors. Benefits of such approach include faster training, higher accuracy of final solutions, identification of a small number of support vectors near decision borders, and efficient handling of classes with small number of vectors. Results on satellite image classification and hypothyroid disease obtained with this type of training are better than any other neural network results published so far.

Among the most important design issues to be addressed to optimize the generalization abilities of trained artificial neural networks (ANNs) are the specific architecture and the composition of the training data set (TDS). Recent work has focused on investigating each of these prerequisites separately. However, some researchers have pointed out the interacting dependencies of ANN topology and the information contained in the TDS. In order to generate coadapted ANNs and TDSs without human intervention we investigate the use of symbiotic (cooperative) coevolution.

A constructive learning algorithm is described that builds a feedforward neural network with an optimal number of hidden units to balance convergence and generalization. The method starts with a small training set and a small network, and expands the training set incrementally after training. If the training does not converge, the network grows incrementally to increase its learning capacity. This process, called selective learning with flexible neural architectures (self), results in a construction of an optimal size network for learning all the given data using only a minimal subset of them. We show that the network size optimization combined with active example selection generalizes significantly better and converges faster than conventional methods. I. Introduction Feedforward neural networks with a hidden layer of sigmoid units are capable of approximating any continuous multivariate function to any desired degree of accuracy [11, 13]. However, this existence theorem does not prov...

The Standard BackPropagation (SBP) algorithm is the most widely known and used learning method for training neural networks. Unfortunately, SBP suffers from several problems such as sensitivity to the initial conditions and very slow convergence. Here we describe how we used Genetic Programming, a search algorithm inspired by Darwinian evolution, to discover new supervised learning algorithms for neural networks which can overcome some of these problems. Comparing our new algorithms with SBP on different problems we show that these are faster, are more stable and have greater feature extracting capabilities.

. A Bayesian framework for genetic programming GP is presented. This is motivated by the observation that genetic programming iteratively searches populations of fitter programs and thus the information gained in the previous generation can be used in the next generation. The Bayesian GP makes use of Bayes theorem to estimate the posterior distribution of programs from their prior distribution and likelihood for the fitness data observed. Offspring programs are then generated by sampling from the posterior distribution by genetic variation operators. We present two GP algorithms derived from the Bayesian GP framework. One is the genetic programming with the adaptive Occam's Z. razor AOR designed to evolve parsimonious programs. The other is the genetic programming with Z. incremental data inheritance IDI designed to accelerate evolution by active selection of fitness cases. A multiagent learning task is used to demonstrate the effectiveness of the presented methods. In a series of experiments, AOR reduced solution complexity by 20% and IDI doubled evolution speed, both without loss of solution accuracy. Keywords: Bayesian genetic programming, probabilistic evolution, adaptive Occam's razor, incremental data inheritance, parsimony pressure, data subset selection 1.