This paper presents the mechanism of Intelligent Adaptive Curiosity. This is an intrinsic motivation system which pushes the robot towards situations in which it maximizes its learning progress. It makes the robot focus on situations which are neither too predictable nor too unpredictable. This mechanism is a source of autonomous mental development for the robot: the complexity of its activities autonomously increases and a developmental sequence appears without being manually constructed. We test this motivation system on a real robot which evolves on a baby play mat with objects that it can learn to manipulate. We show that it first spends time in situations which are easy to learn, then shifts progressively its attention to situations of increasing

Currently, there is a lack of general-purpose, in-place learning engines that incrementally learn multiple tasks, to develop “soft ” multi-task-shared invariances in the intermediate internal representation while a developmental robot interacts with its environment. In-place learning is a biologically inspired concept, rooted in the genomic equivalence principle, meaning that each neuron is responsible for its own development while interacting with its environment. With in-place learning, there is no need for a separate learning network. Computationally, biologically inspired, in-place learning provides unusually efficient learning algorithms whose simplicity, low computational complexity, and generality are set apart from typical conventional learning algorithms. We present in this paper the multiple-layer in-place learning network (MILN) for this ambitious goal. As a key requirement for autonomous mental development, the network enables both unsupervised and supervised learning to occur concurrently, depending on whether motor supervision signals are available or not at the motor end (the last layer) during the agent’s interactions with the environment. We present principles based on which MILN automatically develops invariant neurons in different layers and why such invariant neuronal clusters

What is available to developmental programs in autonomous mental development, and what should be learned at the very early stages of mental development? Our observation is that sensory and motor primitives are the most basic components present at the beginning, and what developmental agents need to learn from these resources is what their internal sensory states stand for. In this paper, we investigate the question in the context of a simple biologically motivated visuomotor agent. We observe and acknowledge, as many other researchers do, that action plays a key role in providing content to the sensory state. We propose a simple, yet powerful learning criterion, that of invariance, where invariance simply means that the internal state does not change over time. We show that after reinforcement learning based on the invariance criterion, the property of action sequence based on an internal sensory state accurately reflects the property of the stimulus that triggered that internal state. That way, the meaning of the internal sensory state can be firmly grounded on the property of that particular action sequence. We expect the framing of the problem and the proposed solution presented in this paper to help shed new light on autonomous understanding in developmental agents such as humanoid robots.

This paper presents a computational theory of developmental mental architectures for artificial and natural systems, motivated by neuroscience. The work is an attempt to approximately model biological mental architectures using mathematical tools. Six types of architecture are presented, beginning with the observation-driven Markov decision process as Type-1. From Type-1 to Type-6, the architecture progressively becomes more complete toward the necessary functions of autonomous mental development. Properties of each type are presented. Experiments are discussed with emphasis on their architectures. r 2007 Published by Elsevier B.V.

The inherent value system of a developmental agent enables autonomous mental development to take place right after the agent’s “birth. ” Biologically, it is not clear what basic components constitute a value system. In the computational model introduced here, we propose that inherent value systems should have at least three basic components: punishment, reward and novelty with decreasing weights from the first component to the last. Punishments and rewards are temporally sparse but novelty is temporally dense. We present a biologically inspired computational architecture that guides development of sensorimotor skills through real-time interactions with the environments, driven by an inborn value system. The inherent value system has been successfully tested on an artificial agent in a simulation environment and a robot in the real world.

Summary. How can agents, natural or artificial, learn about the external environment based only on its internal state (such as the activation patterns in the brain)? There are two problems involved here: first, forming the internal state based on sensory data to reflect reality, and second, forming thoughts and desires based on these internal states. (Aristotle termed these passive and active intellect, respectively [1].) How are these to be accomplished? Chapters in this book consider mechanisms of the instinct for learning (chapter PERLOVSKY) and reinforcement learning (chapter IFTEKHARUDDIN; chapter WERBOS), which modify the mind’s representation for better fitting sensory data. Our approach (as those in chapters FREEMAN and KOZMA) emphasizes the importance of action in this process. Action plays a key role in recovering sensory stimulus properties that are represented by the internal state. Generating the right kind of action is essential to decoding the internal state. Action that maintains invariance in the internal state are important as it will have the same property as that of the represented sensory stimulus. However, such an approach alone does not address how it can be generalized to learn more complex

Abstract not found

The new field of developmental robotics faces the formidable challenge of implementing effective learning mechanisms in complex, dynamic environments. We make a case that knowledge-based learning algorithms might help to meet this challenge. A constructive neural learning algorithm, knowledge-based cascade-correlation (KBCC), autonomously recruits previously-learned networks in addition to the single hidden units recruited by ordinary cascade-correlation. This enables learning by analogy when adequate prior knowledge is available, learning by induction from examples when there is no relevant prior knowledge, and various combinations of analogy and induction. A review of experiments with KBCC indicates that recruitment of relevant existing knowledge typically speeds learning and sometimes enables learning of otherwise impossible problems. Some additional domains of interest to developmental robotics are identified in which knowledge-based learning seems essential. The characteristics of KBCC in relation to other knowledge-based neural learners and analogical reasoning are summarized as is the neurological basis for learning from knowledge. Current limitations of this approach and directions for future work are discussed. Keywords: Knowledge-based learning; neural networks; knowledge transfer; developmental robotics. 245 246 T. R. Shultz et al. 1.

journal homepage: www.elsevier.com/locate/robot Active learning of inverse models with intrinsically motivated goal exploration in