In this work, we describe the evolutionary training of artificial neural network controllers for competitive team game playing behaviors by teams of real mobile robots. This research emphasized the development of methods to automate the production of behavioral robot controllers. We seek methods that do not require a human designer to define specific intermediate behaviors for a complex robot task. The work made use of a real mobile robot colony (EVolutionary roBOTs) and a closely coupled computer-based simulated training environment. The acquisition of behavior in an evolutionary robotics system was demonstrated using a robotic version of the game Capture the Flag. In this game, played by two teams of competing robots, each team tries to defend its own goal while trying to `attack' another goal defended by the other team. Robot neural controllers relied entirely on processed video data for sensing of their environment. Robot controllers were evolved in a simulated environment using evolutionary training algorithms. In the evolutionary process, each generation consisted of a competitive tournament of games played between the controllers in an evolving population. Robot controllers were selected based on whether they won or lost games in the course of a tournament. Following a tournament, the neural controllers were ranked competitively according to how many games they won and the population was propagated using a mutation and replacement strategy. After several hundred generations, the best performing controllers were transferred to teams of real mobile robots, where they exhibited behaviors similar to those seen in simulation including basic navigation, the ability to distinguish between different types of objects, and goal tending behaviors.

Abstract. Evolutionary car racing (ECR) is extended to the case of two cars racing on the same track. A sensor representation is devised, and various methods of evolving car controllers for competitive racing are explored. ECR can be combined with co-evolution in a wide variety of ways, and one aspect which is explored here is the relative-absolute fitness continuum. Systematical behavioural differences are found along this continuum; further, a tendency to specialization and the reactive nature of the controller architecture are found to limit evolutionary progress. 1

In recent years a number of researchers have successfully applied artificial evolution approaches to the design of controllers for autonomous robots. To date, however, Evolutionary Robotics research has focussed almost exclusively on the design of single-robot systems. We are interested in the evolution of controllers for multi-robot systems that are capable of exhibiting cooperative and coordinated behaviour. We report on recent work in which we employed artificial evolution to design neural network controllers for small, homogeneous teams of mobile autonomous robots. The robots are evolved to perform a formation movement task from random starting positions, equipped only with infrared sensors. The dual constraints of homogeneity and minimal sensors make this a non-trivial task. We describe the behaviour of a successful evolved system, in which robots work as a team, adopting and maintaining distinct but interdependent roles in order to achieve the task. We believe this to be the first successful use of evolutionary robotics methodology to develop cooperative, coordinated behaviour for a real multi-robot system.

Abstract not found

We use evolutionary game theory (EGT) to investigate the dynamics and equilibria of selection methods in coevolutionary algorithms. The canonical selection method used in EGT is equivalent to the standard “fitness-proportional” selection method used in evolutionary algorithms. All attractors of the EGT dynamic are Nash equilibria; we focus on simple symmetric variable-sum games that have polymorphic Nash-equilibrium attractors. Against the dynamics of proportional selection, we contrast the behaviors of truncation selection, @ A @ C A, linear ranking, Boltzmann, and tournament selection. Except for Boltzmann selection, each of the methods we test unconditionally fail to achieve polymorphic Nash equilibrium. Instead, we find point attractors that lack game-theoretic justification, cyclic dynamics, or chaos. Boltzmann selection converges onto polymorphic Nash equilibrium only when selection pressure is sufficiently low; otherwise, we obtain attracting limit-cycles or chaos. Coevolutionary algorithms are often used to search for solutions (e.g., Nash equilibria) of games of strategy; our results show that many selection methods are inappropriate for finding polymorphic Nash solutions to variable-sum games. Another application of coevolution is to model other systems; our results emphasize the degree to which the model’s behavior is sensitive to implementation details regarding selection—details that we might not otherwise believe to be critical.

Assume a coevolutionary algorithm capable of storing and utilizing all phenotypes discovered during its operation, for as long as it operates on a problem; that is, assume an algorithm with a monotonically increasing knowledge of the search space. We ask: If such an algorithm were to periodically report, over the course of its operation, the best solution found so far, would the quality of the solution reported by the algorithm improve monotonically over time? To answer this question, we construct a simple preference relation to reason about the goodness of di#erent individual and composite phenotypic behaviors. We then show that whether the solutions reported by the coevolutionary algorithm improve monotonically with respect to this preference relation depends upon the solution concept implemented by the algorithm. We show that the solution concept implemented by the conventional coevolutionary algorithm does not guarantee monotonic improvement; in contrast, the game-theoretic solution concept of Nash equilibrium does guarantee monotonic improvement. Thus, this paper considers 1) whether global and objective metrics of goodness can be applied to coevolutionary problem domains (possibly with open-ended search spaces), and 2) whether coevolutionary algorithms can, in principle, optimize with respect to such metrics and find solutions to games of strategy.

Reports on evolutionary robotics systems have so far been on evolving controllers that make simple robots do simple tasks in simple environments. In this paper we try to stress the evolutionary robotics approach by evolving a controller for a more complex task, namely Khepera robot soccer, and evaluate evolved controller performance against handcoded controllers. We present a system that uses competitive co-evolution to develop robot controllers for the task. The system is described, and performance of the system is documented. Co-evolution is tested against single-population evolution, and it is concluded that co-evolution has the ability to produce more robust individuals with respect to opponent strategies.

Abstract: We propose a modular architecture for autonomous robots which allows for the implementation of basic behavioral modules by both programming and training, and accommodates for an evolutionary development oftheinter-connections among modules. This architecture can implement highly complex controllers and allows for incremental shaping of the robot behavior. Our pro-posal is exempli ed and evaluated experimentally through a number of mobile robotic tasks involving exploration, battery recharging and object manipulation.

We report on recent work in which we employed artificial evolution to design neural network controllers for small, homogeneous teams of mobile autonomous robots. The robots are evolved to perform a formation movement task from random starting positions, equipped only with infrared sensors. The dual constraints of homogeneity and minimal sensors make this a non-trivial task. We describe the behaviour of a successful evolved team in which robots adopt and maintain functionally distinct roles in order to achieve the task. We believe this to be the first example of the use of artificial evolution to design coordinated, cooperative behaviour for real robots.

Co-evolutionary learning involves a training process where training samples are instances of solutions that interact strategically to guide the evolutionary (learning) process. One main research issue is with the generalization performance, i.e., the search for solutions (e.g., input-output mappings) that best predict the required output for any new input that has not been seen during the evolutionary process. However, there is currently no such framework for determining the generalization performance in co-evolutionary learning even though the notion of generalization is well-understood in machine learning. In this paper, we introduce a theoretical framework to address this research issue. We present the framework in terms of game-playing although our results are more general. Here, a strategy’s generalization performance is its average performance against all test strategies. Given that the true value may not be determined by solving analytically a closed-form formula and is computationally prohibitive, we propose an estimation procedure that computes the average performance against a small sample of random test strategies instead. We perform a mathematical analysis to provide a statistical claim on the accuracy of our estimation procedure, which can be further improved by performing a second estimation on the variance of the random variable. For game-playing, it is well-known that one is more interested in the generalization