Abstract. This paper presents a constructive model by which a robot acquires the ability of joint attention with a human caregiver based on its embedded mechanisms of visual attention and learning with self-evaluation. The former is to look at a salient object in the robot’s view, and the latter is to learn sensorimotor co-ordination when visual attention has succeeded. Since the success of visual attention does not always correspond to the success of joint attention, the robot has incorrect learning data for joint attention as well as correct data. However, the robot is expected statistically to lose incorrect data as outliers since such data do not have any correlation in the sensorimotor co-ordination while correct data have a correlation. The robot consequently acquires the ability of joint attention by finding the correlation in the sensorimotor co-ordination even if multiple objects are placed at random positions in an environment and a human caregiver does not provide any task evaluation to the robot. The experimental results show that the proposed model makes the robot reproduce the developmental process of infants ’ joint attention. Therefore, the proposed model could be one of the models to explain how infants develop the ability of joint attention.

We propose a view of gaze following in which infants act as Bayesian learners actively attempting to identify the operating characteristics of the systems with which they interact. We present results of an experiment in which 28 infants (average age 10 months) interacted for a 3 minute period with a non-humanoid robot. For half the infants the robot simulated contingency structure typically produced by human beings. In particular it provided causal information about the existence of a line of regard. For the other 14 infants, the robot behaved in a manner which was not contingent with the environment. We found that a few minutes of interaction with the contingent robot was sufficient to elicit statistically detectable gaze following. There were clear signs that some of these infants were actively attempting to identify whether or not the robot was responsive to them. We propose that the infant brain is equipped to learn and analyze the contingency structure of real-time social interactions. Contingency is a fundamental perceptual dimension used by infants to recognize the operational properties of humans and to generalize existing behaviors to new social partners.

Gaze following is the ability to re-direct one’s gaze to the location where another agent is looking. We present a computational model of how hu-man infants or other agents may acquire gaze following by learning to pre-dict the locations of interesting sights from the looking behavior of other agents through reinforcement learning. The model accounts for many find-ings about the development of gaze following in human infants. During learning, the model develops pre-motor representations that exhibit many properties characteristic of mirror neurons, but they are specific to looking behaviors. The existence of such a new class of mirror neurons is the main prediction of our model. The model also offers a parsimonious account of how these and possibly other mirror neurons may acquire their special re-sponse properties. In this account, visual representations of other agents’ actions become associated with pre-motor neurons that represent the inten-tion to perform corresponding actions. The model also demonstrates how this development may be obstructed in autism spectrum disorder, giving rise to specific physiological and anatomical differences in the mirror system.

Robust joint visual attention is necessary for achieving a common frame of reference between humans and robots interacting multimodally in order to work together on realworld spatial tasks involving objects. We make a comprehensive examination of one component of this process that is often otherwise implemented in an ad hoc fashion: the ability to correctly determine the object referent from deictic reference including pointing gestures and speech. We develop a modular spatial reasoning framework based around decomposition and resynthesis of speech and gesture into a language of pointing and object labeling that supports multimodal and unimodal access in both real-world and mixedreality workspaces, accounts for the need to discriminate and sequence identical and proximate objects, assists in overcoming inherent precision limitations in deictic gesture, and assists in the extraction of those gestures. We further discuss an implementation of the framework that has been deployed on two humanoid robot platforms to date. 1.

It is often asserted that young children’s word learning is guided by constraints or internal biases. Constraints are broadly described as ‘‘any factor that favors some possibilities over others’ ’ (Medin et al., 1990). Researchers have argued that specialized lexical constraints cause children to make some inferences about word meanings before others. An analysis shows that the concept constraint is not informative because it does not differentiate a circumscribed set of word learning behaviors. Defining constraints as innate and domain-specific does not remedy this problem. We cannot separate the effects of so-called constraints or biases from a wide range of cognitive and contextual influences on children’s inferences about novel word meanings. This conclusion is supported by a selective review of these influences. The summary highlights our need for an explanatory framework that is sufficiently rich to capture the flexibility and diversity of children’s word learning. The core of such a framework is summarized as a set of general characteristics of human word learning. These characteristics must serve as a starting point for any viable theory of word learning. Prescriptions for future development of a viable framework are suggested. © 2000 Academic Press Word learning 1 is a complex and intractable problem for which researchers have offered a seemingly simple and powerful solution. The problem is that preschoolers ’ prolific acquisition of new words (averaging a half dozen per day; Carey, 1978) seems impossible given the radical indeterminacy of word meanings. A novel word has an indefinite number of possible meanings, and it is unlikely that children regularly receive information that unambiguously specifies a single meaning. Yet children often infer new words ’ correct or Preparation of the manuscript was supported by a postdoctoral fellowship from the Spencer

Four studies investigated whether and when infants connect information about an actor's affect and perception to their action. Arguably, this may be a crucial way in which infants come to recognize the intentional behaviors of others. In Study 1 an actor grasped one of two objects in a situation where cues from the actor's gaze and expression could serve to determine which object would be grasped, specifically the actor first looked at and emoted positively about one object but not the other. Twelve-month-olds, but not 8-month-olds, recognized that the actor was likely to grasp the object which she had visually regarded with positive affect. Studies 2, 3, and 4 replicated the main finding from Study 1 with 12- and 14-month-olds and included several contrasting conditions and controls. These studies provide evidence that the ability to use information about an adult's direction of gaze and emotional expression to predict action is both present, and developing at the end of the first year of life. q 2002 Elsevier Science B.V. All rights reserved.

Abstract not found

The capacity for shared attention is a cornerstone of human social intelligence. Recent accounts attribute the emergence of shared attention to multiple cognitive mechanisms. Current behavioral data support an alternative dynamic systems model, but many questions remain. To answer these questions and test alternative theories, robotic models will play a critical role. Robotic models reduce the scope of the modeling task, permit comparison of empirically supported theories, and encourage parsimonious models of complex behaviors. Current efforts to model the emergence of shared attention are described. 1.

This book is about the interface between natural language and the sensorimotor system. It is obvious that there is an interface between language and sensorimotor cognition, because we can talk about what we see and do. The main proposal in the book is that the interface is more direct than is commonly assumed. To argue for this proposal I focus on a simple concrete episode—a man grabbing a cup—which can be reported in a simple transitive sentence (e.g. the English sentence The man grabbed a cup). In the first part of the book I present a detailed model of the sensorimotor processes involved in experiencing this episode, both as the agent bringing it about and as an observer watching it happen. The model draws on a large body of research in neuroscience and psychology. I also present a model of the syntactic structure of the associated transitive sentence, developed within the entirely separate discipline of theoretical linguistics. This latter model is a version of Chomsky’s ‘Minimalist ’ syntactic theory, which assumes that a sentence reporting the episode has the same underlying syntactic structure (called ‘logical form’) regardless of which language it is in. My main proposal is that these two independently motivated models are in fact closely

Abstract — We present a reinforcement learning model of gaze following that learns to incorporate eye cues in a developmental trajectory similar to that found in infants: older infants follow gaze more frequently when both the caregiver’s eyes and head are turned than when only her head is turned. Similarly, infants learn to follow gaze less when the caregiver’s eyes are closed than when her eyes are open. The model works through the maximization of visual reward, and not by representing an estimate of the other person’s attentional state, as would be expected by adherents of a mentalist interpretation of gaze following. Within the debate about the age of onset in the use of eye cues for gaze following, we hypothesize that eye cues have an effect as soon as the infant learns to follow gaze, but at first it might be small and therefore difficult to measure. Finally, we propose this learning approach for modeling other aspects of theory of mind. Index Terms — gaze following, joint attention, theory of mind, reinforcement learning, temporal difference learning, actor critic architecture. I.