This paper presents a skill learning model CLARION. Different from existing models of mostly high-level skill learning that use a top-down approach (that is, turning declarative knowledge into procedural knowledge through practice), we adopt a bottom-up approach toward low-level skill learning, where procedural knowledge develops first and declarative knowledge develops later. Our model is formed by integrating connectionist, reinforcement, and symbolic learning methods to perform on-line reactive learning. It adopts a two-level dual-representation framework (Sun, 1995), with a combination of localist and distributed representation. We compare the model with human data in a minefield navigation task, demonstrating some match between the model and human data in several respects.

Abstract—Exploratory activities seem to be intrinsically rewarding for children and crucial for their cognitive development. Can a machine be endowed with such an intrinsic motivation system? This is the question we study in this paper, presenting a number of computational systems that try to capture this drive towards novel or curious situations. After discussing related research coming from developmental psychology, neuroscience, developmental robotics, and active learning, this paper presents the mechanism of Intelligent Adaptive Curiosity, an intrinsic motivation system which pushes a robot towards situations in which it maximizes its learning progress. This drive makes the robot focus on situations which are neither too predictable nor too unpredictable, thus permitting autonomous mental development. The complexity of the robot’s activities autonomously increases and complex developmental sequences self-organize without

A theoretical framework for grounding language is introduced that provides a computational path from sensing and motor action to words and speech acts. The approach combines concepts from semiotics and schema theory to develop a holistic approach to linguistic meaning. Schemas serve as structured beliefs that are grounded in an agent’s physical environment through a causal-predictive cycle of action and perception. Words and basic speech acts are interpreted in terms of grounded schemas. The framework reflects lessons learned from implementations of several language processing robots. It provides a basis for the analysis and design of situated, multimodal communication systems that straddle symbolic and non-symbolic realms.

: This paper presents a novel learning model Clarion, which is a hybrid model based on the twolevel approach proposed in Sun (1995). The model integrates neural, reinforcement, and symbolic learning methods to perform on-line, bottom-up learning (i.e., learning that goes from neural to symbolic representations). The model utilizes both procedural and declarative knowledge (in neural and symbolic representations respectively), tapping into the synergy of the two types of processes. It was applied to deal with sequential decision tasks. Experiments and analyses in various ways are reported that shed light on the advantages of the model. obstacles agent target Figure 1: Navigating Through A Minefield 1 Introduction This paper presents a model that unifies neural, symbolic, and reinforcement learning. It addresses the following three issues: (1) It deals with autonomous learning: It allows a situated agent to learn autonomously and continuously, from on-going experience in the world, w...

What do the first concepts look like? I propose that the earliest concepts learned by infants are abstractions of activities. The semantics of these concepts are predictive---a good abstraction is one that will help the infant predict reward. This idea has been implemented in several programs, in particular, as fluents in the Babysimulator and preimages in Coelho and Grupen 's robotics work. Additional examples and a longer version of this paper can be found at http://eksl-www.cs.umass.edu /research/conceptual-systems/index.html

Shaping by successive approximations is an important animal training technique in which behavior is gradually adjusted in response to strategically timed reinforcements. We describe a computational model of this shaping process and its implementation on a mobile robot. Innate behaviors in our model are sequences of actions and enabling conditions, and shaping is a behavior editing process realized by multiple editing mechanisms. The model replicates some fundamental phenomena associated with instrumental learning in animals, and allows an RWI B21 robot to learn several distinct tasks derived from the same innate behavior. 1. Introduction Service dogs trained to assist a disabled person will respond to over 60 verbal commands to, for example, turn on lights, open a refrigerator door, or retrieve a dropped object [9]. Chicks can be taught to play a toy piano (peck out a key sequence until a reinforcement is received at the end of the tune) [6], and rats have been conditioned to perform c...

This paper discusses the concept of joint attention and the di#erent skills underlying its development. We argue that joint attention is much more than gaze following or simultaneous looking because it implies a shared intentional relation to the world. The current state-of-the-art in robotic and computational models of the di#erent prerequisites of joint attention is discussed in relation with a developmental timeline drawn from results in child studies.

To operate e#ectively in complex environments learning agents have to selectively ignore irrelevant details by forming useful abstractions. In this article we outline a formulation of abstraction for reinforcement learning approaches to stochastic sequential decision problems modeled as semiMarkov Decision Processes (SMDPs). Building on existing algebraic approaches, we propose the concept of SMDP homomorphism and argue that it provides a useful tool for a rigorous study of abstraction for SMDPs. We apply this framework to di#erent classes of abstractions that arise in hierarchical systems and discuss relativized options, a framework for compactly specifying a related family of temporally-extended actions. Additional details of this work are described in refs. [1, 2, 3].

Eric B. Baum NEC Research Institute 4 Independence Way Princeton, NJ 08540 eric@research.nj.nec.com Abstract. I argue that the mind should be viewed as an economy, and describe an algorithm that autonomously apportions complex tasks to multiple cooperating agents in such a way that the incentive of each agent is exactly to maximize my reward, as owner of the system. A specific model, called "The Hayek Machine" is proposed and tested on a simulated Blocks World (BW) planning problem. Hayek learns to solve far more complex BW problems than any previous learning algorithm. If given intermediate reward and simple features, it learns to efficiently solve arbitrary BW problems. 1 Introduction I am interested in understanding how human-like mental capabilities can arise. Any such understanding must model how large computational tasks can be broken down into smaller components, how such components can be coordinated, how the system can gain knowledge, how computations performed can be trac...

This paper introduces TABASCO, an architecture for software agents aimed at integrating results from functional theories in emotion research and insights on the impact of the capacities and limitations of perception in a framework orientated along the situated "New AI"/ALife approach. This expository paper first briefly summarizes current views on the nature and function of emotion and then discusses related current appraisal theories in more detail. A survey of existing approaches to emotion synthesis is followed by a first outline of the TABASCO architecture, relating it to the areas of research in psychology, ALife and agent architectures.