Adaptive Execution in Complex Dynamic Worlds Robert James Firby Yale University 1989 A robot acting in the real world must use flexible plans because actions will sometimes fail to produce desired effects, and unexpected events will sometimes demand the robot shift its attention. A plan is usually construed as a list of primitive robot actions to be executed one after another but in a complex domain, a plan must be structured to cope effectively with the myriad unpredictable details it will encounter during execution. However, adding structure to a plan involves more than augmenting the primitive plan representation; it requires a complete model of interaction with the world called situation-driven execution. Situation-driven execution assumes that a plan consists of tasks with three major components: a satisfaction test, a window of activity, and a set of execution methods that are appropriate in different circumstances. Execution of such a plan proceeds by selecting an unsatisfied t...

cockpit interface Abstract cockpit interface Abstract cockpit interface Abstract cockpit interface Figure 1: Human and automated pilots interact with the DIS environment via distributed simulators.

The paper describes the results of the evolutionary development of a real, neural-network driven mobile robot. The evolutionary approach tothe development of neural controllers for autonomous agents has been successfully used by many researchers, but most-if not all- studies have been carried out with computer simulations. Instead, in this research the whole evolutionary process takes places entirely on a real robot without human intervention. Although the experiments described here tackle a simple task of navigation and obstacle avoidance, we show a number of emergent phenomena that are characteristic of autonomous agents. The neural controllers of the evolved best individuals display a full exploitation of non-linear and recurrent connections that make them more e cient than analogous man-designed agents. In order to fully understand and describe the robot behavior, we have also employed quantitative ethological tools [13], and showed that the adaptation dynamics conform to predictions made for animals. 1

The interaction of an autonomous mobile robot with the real world critically depends on the robots morphology and on its environment. Building a model of these aspects is extremely complex, making simulation insufficient for accurate validation of control algorithms. If simulation environments are often very efficient, the tools for experimenting with real robots are often inadequate. The traditional programming languages and tools seldom provide enought support for realtime experiments, thus hindering the understanding of the control algorithms and making the experimentation complex and time-consuming. A miniature robot is presented: it has a cylindrical shape measuring 55 mm in diameter and 30 mm in height. Due to its small size, experiments can be performed quickly and cost-effectively in a small working area. Small peripherals can be designed and connected to the basic module and can take advantage of a versatile communication scheme. A serial-link is provided to run control algorithms on a workstation during debugging, thereby giving the user the opportunity of employing all available graphical tools. Once debugged, the algorithm can be downloaded to the robot and run on its own processor. Experimentation with groups of robots is hardly possible with commercially available hardware. The size and the price of the described robot open the way to cost-effective investigations into collective behaviour. This aspect of research drives the design of the robot described in this paper. Experiments with some twenty units are planned for the near future.

This thesis addresses situated, embodied agents interacting in complex domains. It focuses on two problems: 1) synthesis and analysis of intelligent group behavior, and 2) learning in complex group environments. Basic behaviors, control laws that cluster constraints to achieve particular goals and have the appropriate compositional properties, are proposed as effective primitives for control and learning. The thesis describes the process of selecting such basic behaviors, formally specifying them, algorithmically implementing them, and empirically evaluating them. All of the proposed ideas are validated with a group of up to 20 mobile robots using a basic behavior set consisting of: safe--wandering, following, aggregation, dispersion, and homing. The set of basic behaviors acts as a substrate for achieving more complex high--level goals and tasks. Two behavior combination operators are introduced, and verified by combining subsets of the above basic behavior set to implement collective flocking, foraging, and docking. A methodology is introduced for automatically constructing higher--level behaviors

firestorm of interest in autonomous robots with the introduction of the Subsumption architecture 1 [Brooks86]. At the time, the dominant view in the AI community was that a control system for an autonomous mobile robot should be decomposed into three functional elements: a sensing system, a planning system, and an execution system [Nilsson80]. The job of the sensing system is to translate raw sensor input (usually sonar or vision data) into a world model. The job of the planner is to take the world model and a goal and generate a plan to achieve the goal. The job of the execution system is to take the plan and generate the actions it prescribes. The sense-plan-act (SPA) approach has two significant architectural features. First, the flow of

Abstract. To explore issues of developmental structure, physical embodiment, integration of multiple sensory and motor systems, and social interaction, we have constructed an upper-torso humanoid robot called Cog. The robot has twenty-one degrees of freedom and a variety of sensory systems, including visual, auditory, vestibular, kinesthetic, and tactile senses. This chapter gives a background on the methodology that we have used in our investigations, highlights the research issues that have been raised during this project, and provides a summary of both the current state of the project and our long-term goals. We report on a variety of implemented visual-motor routines (smooth-pursuit tracking, saccades, binocular vergence, and vestibular-ocular and opto-kinetic reflexes), orientation behaviors, motor control techniques, and social behaviors (pointing to a visual target, recognizing joint attention through face and eye finding, imitation of head nods, and regulating interaction through expressive feedback). We further outline a number of areas for future research that will be necessary to build a complete embodied system. 1

this paper, I will argue that, contrary to these challenges, planning deserves its central place on the AI map. I will claim that intelligent agents are planning agents, and that philosophical and commonsense psychological theorizing about the process of planning can provide useful insights into the question of agent design. The theories I have in mind are not restricted to The Uses of Plans 3 how agents can form plans. Much of my research has concerned the ways in which intelligent agents use their plans. I will describe some of that research, and will argue that plans are used not only to guide action, but also to control reasoning and to enable inter-agent coordination. These uses of plans make possible intelligent behavior in complex, dynamic, multiagent environments. 2 Planning We can begin by asking what exactly we mean by "planning". For many years, planning had a quite specific meaning in AI: it was the process of formulating a program of action to achieve some specified goal. You gave a planning system a description of initial conditions and a goal, and it produced a plan of action whose execution in a state satisfying the initial conditions was guaranteed to result in a state satisfying the goal. These plans were akin to recipes for achieving the goal. Your goal might be to have a chocolate cake. In the initial state, you might have eggs, milk, and chocolate, a pan and a working oven. In these conditions, a valid plan might be to go the store to buy some flour, return home, preheat the oven, mix the ingredients, pour the mixture into the pan, and put it in the oven for 45 minutes. Traditional AI planning systems like STRIPS [22], NOAH [63], and SIPE [71], were designed to construct just this kind of plan---except usually the goal was something like a tower o...

This paper illustrates an artificial developmental system that is a computationally efficient technique for the automatic generation of complex Artificial Neural Networks (ANN). Artificial developmental system can develop a graph grammar into a modular ANN made of a combination of more simple subnetworks. A genetic algorithm is used to evolve coded grammars that generates ANNs for controlling a six-legged robot locomotion. A mechanism for the automatic definition of sub-neural networks is incorporated. Using this mechanism, the genetic algorithm can automatically decompose a problem into subproblems, generate a subANN for solving the subproblem, and instantiate copies of this subANN to build a higher level ANN that solves the problem. We report some simulation results showing that the same problem cannot be solved if the mechanism for automatic definition of sub-networks is suppressed. We support our argumentation with pictures describing the steps of development, how ANN structures ar...

. This article presents a new reinforcement learning method called SANE (Symbiotic, Adaptive Neuro-Evolution), which evolves a population of neurons through genetic algorithms to form a neural network capable of performing a task. Symbiotic evolution promotes both cooperation and specialization, which results in a fast, efficient genetic search and discourages convergence to suboptimal solutions. In the inverted pendulum problem, SANE formed effective networks 9 to 16 times faster than the Adaptive Heuristic Critic and 2 times faster than Q- learning and the GENITOR neuro-evolution approachwithout loss of generalization. Such efficient learning, combined with few domain assumptions, make SANE a promising approach to a broad range of reinforcement learning problems, including many real-world applications. Keywords: Neuro-Evolution, Reinforcement Learning, Genetic Algorithms, Neural Networks. 1. Introduction Learning effective decision policies is a difficult problem that appears in m...