Developmental robotics is an emerging field located at the intersection of robotics, cognitive science and developmental sciences. This paper elucidates the main reasons and key motivations behind the convergence of fields with seemingly disparate interests, and shows why developmental robotics might prove to be beneficial for all fields involved. The methodology advocated is synthetic and two-pronged: on the one hand, it employs robots to instantiate models originating from developmental sciences; on the other hand, it aims to develop better robotic systems by exploiting insights gained from studies on ontogenetic development. This paper gives a survey of the relevant research issues and points to some future research directions.

The value system of a developmental robot signals the occurrence of salient sensory inputs, modulates the mapping from sensory inputs to action outputs, and evaluates candidate actions. In the work reported here, a low level value system is modeled and imple- mented. It simulates the non-associative an- imal learning mechanism known as habituation effect. Reinforcement learning is also integrated with novelty. Experimental results show that the proposed value system works as designed in a study of robot viewing angle selection.

Developmental robotics is an emerging field located at the intersection of developmental psychology and robotics, that has lately attracted quite some attention. This paper gives a survey of a variety of research projects dealing with or inspired by developmental issues, and outlines possible future directions.

Abstract—Scaffolding is a process of transferring learned skills to new and more complex tasks through arranged experience in open-ended development. In this paper, we propose a developmental learning architecture that enables a robot to transfer skills acquired in early learning settings to later more complex task settings. We show that a basic mechanism that enables this transfer is sequential priming combined with attention, which is also the driving mechanism for classical conditioning, secondary conditioning, and instrumental conditioning in animal learning. A major challenge of this work is that training and testing must be conducted in the same program operational mode through online, real-time interactions between the agent and the trainers. In contrast with former modeling studies, the proposed architecture does not require the programmer to know the tasks to be learned and the environment is uncontrolled. All possible perceptions and actions, including the actual number of classes, are not available until the programming is finished and the robot starts to learn in the real world. Thus, a predesigned task-specific symbolic representation is not suited for such an open-ended developmental process. Experimental results on a robot are reported in which the trainer shaped the behaviors of the agent interactively, continuously, and incrementally through verbal commands and other sensory signals so that the robot learns new and more complex sensorimotor tasks by transferring sensorimotor skills learned in earlier periods of open-ended development. Index Terms—Attention, classical conditioning, incremental learning, instrumental conditioning, mental architecture, mental development, multitask learning, online learning, scaffolding, skill transfer. I.

The inherent value system of a developmental agent enables autonomous mental development to take place right after the agent’s “birth. ” Biologically, it is not clear what basic components constitute a value system. In the computational model introduced here, we propose that inherent value systems should have at least three basic components: punishment, reward and novelty with decreasing weights from the first component to the last. Punishments and rewards are temporally sparse but novelty is temporally dense. We present a biologically inspired computational architecture that guides development of sensorimotor skills through real-time interactions with the environments, driven by an inborn value system. The inherent value system has been successfully tested on an artificial agent in a simulation environment and a robot in the real world.

Biological organisms display an amazing ability during their ontogenetic development to adaptively develop solutions to the various problems of survival that their environments present to them. Dynamical and embodied models of cognition (Clark, 1997; Edelman & Tononi, 2000; Franklin, 1995; Freeman, 1999a, 1999b; Freeman & Kozma, 2000; Freeman, Kozma, & Werbos, 2000; Hendriks-Jansen, 1996; Kelso, 1995; Kozma & Freeman, 2001; Port & van Gelder, 1995; Skarda & Freeman, 1987; Thelen & Smith, 1994) are beginning to offer new insights into how the numerous, heterogeneous elements of neural structures may self-organize during the development of the organism in order to effectively form adaptive categories and increasingly sophisticated skills, strategies and goals. In this paper we present models of ontogenetic development built on neurologically inspired, bottom-up, dynamic approaches to embodied category formation such as those done by Freeman (1975, 1999b), Freeman and Kozma (2000), Kozma and Freeman (2001), Verschure (1998) and Edelman (1987, 1989). We believe that building on such mechanisms from an embodied dynamical perspective will produce autonomous agents that display greatly increased flexibility in their behavior. Such models will represent a better understanding of how the brains of biological organisms not only form perceptual categories of their environments during development, but also develop effective patterns of behavior through the dynamic self-organization of neurological patterns of activity.

Abstract—There exists a large conceptual gap between symbolic models and emergent models for the mind. Many emergent models work on low-level sensory data, while many symbolic models deal with high-level abstract (i.e., action) symbols. There has been relatively little study on intermediate representations, mainly because of a lack of knowledge about how representations fully autonomously emerge inside the closed brain skull, using information from the exposed two ends (the sensory end and the motor end). As reviewed here, this situation is changing. A fundamental challenge for emergent modelsisabstraction,which symbolic models enjoy through human handcrafting. The term abstract refers to properties disassociated with any particular form. Emergent abstraction seems possible, although the brain appears to never receive a computer symbol (e.g., ASCII code) or produce such a symbol. This paper reviews major agent models with an emphasis on representation. It suggests two different ways to relate symbolic representations with emergent representations: One is based on their categorical definitions. The other considers that a symbolic representation corresponds to a brain’s outside behaviors observed and handcrafted by other outside human observers; but an emergent representation is inside the brain. Index Terms—Agents, attention, brain architecture, complexity, computer vision, emergent representation, graphic models, mental

Abstract—Although reinforcement learning has been extensively modeled, few agent models that incorporate values use biologically plausible neural networks as a uniform computational architecture. We call biologically plausible neural network architecture neuromorphic. This paper discusses some theoretical constraints on neuromorphic intrinsic value systems [3]. By intrinsic, we mean a value system that is likely programmed by the genes, whose value bias has already taken a shape at the birth time. Such an intrinsic value system plays an important role in developing extrinsic values through the agent’s own experience during its life span. Based on our theoretical constraints, we model two types of neurotransmitters, serotonin and dopamine, to construct a neuromorpic intrinsic value system based on a uniform neural network architecture. Serotonin represents punishment and stress, while dopamine represents reward and pleasure. Experimentally, this model allows our simulated robot to develop an attachment to one entity and fear another. I.

Abstract—Many studies have been performed to train a classification network using supervised learning. In order to enable a recognition network to learn autonomously or to later improve its recognition performance through simpler confirmation or rejection, it is desirable to model networks that have an intrinsic motivation system. Although reinforcement learning has been extensively studied, much of the existing models are symbolic whose internal nodes have preset meanings from a set of handpicked symbolic set that is specific for a given task or domain. Neural networks have been used to automatically generate internal (distributed) representations. However, modeling a neuromorphic motivational system for neural networks is still a great challenge. By neuromorphic, we mean that the motivational system for a neural network must be also a neural network, using a standard type of neuronal computation and neuronal learning. This work proposes a neuromorphic motivational system, which includes two subsystems — the serotonin system and the dopamine system. The former signals a large class of stimuli that are intrinsically aversive (e.g., stress or pain). The latter signals a large class of stimuli that are intrinsically appetitive (e.g., sweet and pleasure). We experimented with this motivational system for visual recognition settings to investigate how such a system can learn through interactions with a teacher, who does not give answers, but only punishments and rewards. I.

Abstract not found