Evolvable hardware (EHW) has attracted increasing attention since early 1990's with the advent of easily reconfigurable hardware such as field programmable gate arrays (FPGAs). It promises to provide an entirely new approach to complex electronic circuit design and new adaptive hardware. EHW has been demonstrated to be able to perform a wide range of tasks from pattern recognition to adaptive control. However, there are still many fundamental issues in EHW which remain open. This paper reviews the current status of EHW, discusses the promises and possible advantages of EHW, and indicates the challenges we must meet in order to develop practical and large-scale EHW. 1 Introduction Evolvable hardware (EHW) refers to hardware that can change its architecture and behaviour dynamically and autonomously by interacting with its environment. At present, almost all EHW uses an evolutionary algorithm (EA) as their main adaptive mechanism. One of the key motivations behind EHW is to learn from N...

Abstract. The first successful FPGA implementation [1] of artificial neural networks (ANNs) was published a little over a decade ago. It is timely to review the progress that has been made in this research area. This brief survey provides a taxonomy for classifying FPGA implementations of ANNs. Different implementation techniques and design issues are discussed. Future research trends are also presented. 1

Biological organisms are among the most intricate structures known to man, exhibiting highly complex behavior through the massively parallel cooperation of numerous relatively simple elements, the cells. As the development of computing systems approaches levels of complexity such that their synthesis begins to push the limits of human intelligence, engineers are starting to seek inspiration in nature for the design of computing systems, both at the software and at the hardware levels. This paper will present one such endeavor, notably an attempt to draw inspiration from biology in the design of a novel digital circuit: a field-programmable gate array (FPGA). This reconfigurable logic circuit will be endowed with two features motivated and guided by the behavior of biological systems: self-replication and self-repair. 1

. This paper presents a hardware-friendly approach for adapting the structure of a reinforcement, learning-based neurocontroller. An unsupervised clustering algorithm is used to partition the state space of a system and to adapt the size of its reinforcement module. In the wellknown inverted pendulum problem, the system has proven to be much faster than previous neurocontroller approaches. We are currently working on an implementation of the system using field-programmable logic devices. 1 Introduction A major problem in nonlinear control is the tuning and adaptation of the controller. For this purpose a model of the process is usually developed. Then, following an approximation of the inverse relation between the desired outputs and the control actions, the controller is adjusted. However, for many real-world problems there is no available quantitative data regarding input-output relations, rendering analytical modeling very difficult [6]; furthermore, errors in the model can lead to...

Reinforcement learning is a computational approach to learning from interaction. Tabular implementations of reinforcement learning methods are the most simple, though, they suffer from the curse of dimensionality problem; therefore, function approximation techniques have been used to provide generalization across states and actions. Nevertheless, eligibility traces and integrated learning, planning and reacting methods have been proposed to deal with this problem, and allow us to use tabular implementations. In this paper, we present an empirical comparison between eligibility traces and integrated learning, planning and reacting methods in both Markovian and non-Markovian problems. 1 Introduction A major problem in nonlinear control is the tuning and adaptation of the controller. For this purpose a model of the process is usually developed. Then, following an approximation of the inverse relation between the desired outputs and the control actions, the controller is adjusted. However...

An autonomous robot need not be given all the details of the environment in which it is going to act: it can acquire them by direct interaction. One approach to learn by interaction is reinforcement learning, though, the robot has also to be able to autonomously categorize the input data it receives from the environment, deal with the stability-plasticity dilemma, and learn very rapidly. In this paper we present a digital artificial brain architecture capable of dealing with such problems. Furthermore, we present its use for controlling a mobile autonomous robot in an obstacle avoidance task in a real arena. Keywords. Artificial neural networks, mobile autonomous robots, neurocontrol. 1 Introduction Programming an autonomous robot so that it reliably acts in an unknown or a dynamic environment is a difficult thing to do. This is due to missing information during programming, the dynamic nature of the environment and the inherent noise in the robot's sensors and actuators [1]. One com...

. A well known model of reinforcement learning is called Adaptive Heuristic Critic learning. It is composed of two so called "neuronlike adaptive elements" and has been used to solve difficult learning control problems. In this paper we present an FPGA design and implementation of such algorithm, and, furthermore, we describe a neurocontroller system composed of a network of neuronlike adaptive elements and an unsupervised clustering system called FAST, which dynamically partitions the input state space of the system being controlled. 1 Introduction The Adaptive Heuristic Critic (AHC) algorithm is an early model of reinforcement learning presented by Barto, Sutton, and Anderson in a very influential paper of 1983 entitled "Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems" [2]. Such algorithm has proven to be very successful in solving control problems like the well-known inverted pendulum problem. In this paper we present an FPGA implementation of such a...

This paper describes a networked FPGA-based implementation of the FAST (Flexible Adaptable-Size Topology) architecture, a Artificial Neural Network (ANN) that dynamically adapts its size. Most ANN models base their ability to adapt to problems on changing the strength of the interconnections between computational elements according to a given learning algorithm. However, constrained interconnection structures may limit such ability. Field programmable hardware devices are very well adapted for the implementation of ANN with in-circuit structure adaptation. To realize this implementation we used a network of Labomat 3 boards (a reconfigurable platform developed in our laboratory), which communicate with each other using TCP/IP or a faster, direct hardware connection. 1 Introduction Artificial neural network models offer an attractive paradigm: learning to solve problems from examples. They achieve fast parallel processing via massively parallel non-linear computational elements. While...

Abstract not found

Neurocontrol is a crucial area of fundamental research within the neural network field. Adaptive Heuristic Critic learning is a key algorithm for real time adaptation in neurocontrollers. In this paper we present how an unsupervised neural network model with adaptable structure can be used to speedup Adaptive Heuristic Critic learning, its FPGA design, and how it adapts the neurocontroller to the state space of the system being controlled. 1 Introduction Artificial neural networks are massively parallel systems with the capability of infer a response to an unknown situation, achieved through generalizing previous encountered, known situations. The lack of knowledge in determining the appropriate topology of an artificial neural network, limits such capability. Evolutionary techniques (i.e. genetic algorithms) have been widely used for the design of these networks [2]. Essentially, the algorithm employs a population of neural networks, each encoded as a bit string "genome"; evolution p...