. Lazy learning methods like the k-nearest neighbor classifier require storing the whole training set and may be too costly when this set is large. The condensed nearest neighbor classifier incrementally stores a subset of the sample, thus decreasing storage and computation requirements. We propose to train multiple such subsets and take a vote over them, thus combining predictions from a set of concept descriptions. We investigate two voting schemes: simple voting where voters have equal weight and weighted voting where weights depend on classifiers' confidences in their predictions. We consider ways to form such subsets for improved performance: When the training set is small, voting improves performance considerably. If the training set is not small, then voters converge to similar solutions and we do not gain anything by voting. To alleviate this, when the training set is of intermediate size, we use bootstrapping to generate smaller training sets over which we train the voters. Wh...

Learning when limited to modification of some parameters has a limited scope; the capability to modify the system structure is also needed to get a wider range of the learnable. In the case of artificial neural networks, learning by iterative adjustment of synaptic weights can only succeed if the network designer predefines an appropriate network structure, i.e., number of hidden layers, units, and the size and shape of their receptive and projective fields. This paper advocates the view that the network structure should not, as usually done, be determined by trial-and-error but should be computed by the learning algorithm. Incremental learning algorithms can modify the network structure by addition and/or removal of units and/or links. A survey of current connectionist literature is given on this line of thought. "Grow and Learn" (GAL) is a new algorithm that learns an association at one-shot due to being incremental and using a local representation. During the so-called...

. Artificial neural networks achieve fast parallel processing via massively parallel non-linear computational elements. Most neural network models base their ability to adapt to problems on changing the strength of the interconnections between computational elements according to a given learning algorithm. However, constrained interconnection structures may limit such ability. Field programmable hardware devices allow the implementation of neural networks with in-circuit structure adaptation. This paper describes an FPGA implementation of the FAST (Flexible Adaptable-Size Topology) architecture, a neural network that dynamically changes its size. Since initial experiments indicated a good performance on pattern clustering tasks, we have applied our dynamicstructure FAST neural network to an image segmentation and recognition problem. 1 Introduction Artificial neural network models offer an attractive paradigm: learning to solve problems from examples. Most neural network models base ...

. This paper presents a hardware-friendly approach for adapting the structure of a reinforcement, learning-based neurocontroller. An unsupervised clustering algorithm is used to partition the state space of a system and to adapt the size of its reinforcement module. In the wellknown inverted pendulum problem, the system has proven to be much faster than previous neurocontroller approaches. We are currently working on an implementation of the system using field-programmable logic devices. 1 Introduction A major problem in nonlinear control is the tuning and adaptation of the controller. For this purpose a model of the process is usually developed. Then, following an approximation of the inverse relation between the desired outputs and the control actions, the controller is adjusted. However, for many real-world problems there is no available quantitative data regarding input-output relations, rendering analytical modeling very difficult [6]; furthermore, errors in the model can lead to...

An autonomous robot need not be given all the details of the environment in which it is going to act: it can acquire them by direct interaction. One approach to learn by interaction is reinforcement learning, though, the robot has also to be able to autonomously categorize the input data it receives from the environment, deal with the stability-plasticity dilemma, and learn very rapidly. In this paper we present a digital artificial brain architecture capable of dealing with such problems. Furthermore, we present its use for controlling a mobile autonomous robot in an obstacle avoidance task in a real arena. Keywords. Artificial neural networks, mobile autonomous robots, neurocontrol. 1 Introduction Programming an autonomous robot so that it reliably acts in an unknown or a dynamic environment is a difficult thing to do. This is due to missing information during programming, the dynamic nature of the environment and the inherent noise in the robot's sensors and actuators [1]. One com...

We compare kernel estimators, single and multi-layered perceptrons and radial-basis functions for the problems of classification of handwritten digits and speech phonemes. By taking two different applications and employing many techniques, we report here a twodimensional study whereby a domain-independent assessment of these learning methods can be possible. We consider a feed-forward network with one hidden layer. As examples of the local methods, we use kernel estimators like k-nearest neighbor (k-nn), Parzen windows, generalized k-nn, and Grow and Learn (Condensed Nearest Neighbor). We have also considered fuzzy k-nn due to its similarity. As distributed networks, we use linear perceptron, pairwise separating linear perceptron, and multilayer perceptrons with sigmoidal hidden units. We also tested the radial-basis function network which is a combination of local and distributed networks. Four criteria are taken for comparison: Correct classification of the test set, network size, l...

Neurocontrol is a crucial area of fundamental research within the neural network field. Adaptive Heuristic Critic learning is a key algorithm for real time adaptation in neurocontrollers. In this paper we present how an unsupervised neural network model with adaptable structure can be used to speedup Adaptive Heuristic Critic learning, its FPGA design, and how it adapts the neurocontroller to the state space of the system being controlled. 1 Introduction Artificial neural networks are massively parallel systems with the capability of infer a response to an unknown situation, achieved through generalizing previous encountered, known situations. The lack of knowledge in determining the appropriate topology of an artificial neural network, limits such capability. Evolutionary techniques (i.e. genetic algorithms) have been widely used for the design of these networks [2]. Essentially, the algorithm employs a population of neural networks, each encoded as a bit string "genome"; evolution p...

Most neural network models base their ability to adapt to problems on changing their interconnection strengths according to a learning algorithm. Evolutionary technics and a special class of learning algorithms enable a neural network to have a dynamic structure too. While in the first case we obtain an optimized a-priori architecture the latter allows on-line adaptation. However, most of those algorithms are computationally intensive and difficult to implement in hardware. This paper describes a fully digital implementation of a neural network with on-line automatic structure optimization. 1 Introduction Artificial neural networks are widely used for the design and analysis of adaptive, intelligent systems for a number of reasons including : potential for massively parallel computation, robustness in the presence of noise, resilience to the failure of components, amenability to adaptation and learning (by the modification of computational structures employed), and resemblance to biol...

One of the central problems in the application of neural networks is finding the optimal network topology. This paper introduces the FAST architecture (flexible adaptable-size topology), an on-line, evolving neural network that dynamically adapts its topology through interactions with a problem-specific environment. We present a fully digital implementation of the network and demonstrate its viability on a pattern clustering task. We believe the FAST architecture holds potential by offering a fast, flexible platform for neural network applications. 1: Introduction One of the central problems in the application of neural networks is finding the optimal network topology. In recent years evolutionary techniques, such as genetic algorithms, have been applied to this problem. There are two basic approaches that are taken: (1) the off-line approach, in which the topology is computed apriori, henceforth remaining fixed, and (2) the on-line approach in which the topology dynamically changes as...

. There is a growing evidence that the human brain follows an environmentally-guided neural circuit building that increases its learning flexibility. Similarly, it has been shown that artificial neural networks with dynamic topologies attempt to overcome the problem of determining the appropriate topology to optimally solve a given application. This paper presents a modular structure-adaptable artificial neural network architecture for autonomous control systems consisting of an unsupervised learning network, a reinforcement learning module and a planning module. Finally, we present an extension of the state representation of the environment by introducing short-term memories to deal with the problem of partial observability in the real-world. Keywords: Artificial neural networks, topology adaptation, reinforcement learning, neurocontrol, autonomous mobile robots, partially observable environments. 1 Introduction According to "neural constructivism" the representational features in th...