We introduce an approach for physically-grounded natural language interpretation by robots which reacts appropriately to unanticipated physical changes in the environment and dynamically assimilates new information pertinent to ongoing tasks. At the core of the approach is a model of object schemas that enables a robot to encode beliefs about physical objects in its environment using collections of coupled processes responsible for sensorimotor interaction. These interaction processes run concurrently in order to ensure responsiveness to the environment, while coordinating sensorimotor expectations, action planning, and language use. The model has been implemented on a robot that manipulates objects on a tabletop in response to verbal input. The implementation responds to verbal requests such as “Group the green block and the red apple, ” while adapting in real-time to unexpected physical collisions and taking opportunistic advantage of any new information it may receive through perceptual and linguistic channels.

Abstract — This paper explores methods and representations that allow two perceptually heterogeneous robots, each of which represents concepts via grounded properties, to transfer knowledge despite their differences. This is an important issue, as it will be increasingly important for robots to communicate and effectively share knowledge to speed up learning as they become more ubiquitous. We use Gärdenfors ’ conceptual spaces to represent objects as a fuzzy combination of properties such as color and texture, where properties themselves are represented as Gaussian Mixture Models in a metric space. We then use confusion matrices that are built using instances from each robot, obtained in a shared context, in order to learn mappings between the properties of each robot. These mappings are then used to transfer a concept from one robot to another, where the receiving robot was not previously trained on instances of the objects. We show in a 3D simulation environment that these models can be successfully learned and concepts can be transferred between a ground robot and an aerial quadrotor robot. I.

Abstract. In this paper we present a computational model of lexicon formation that extends the nature of form-meaning associations from previous language game models to be more “flexible”. The model more easily copes with complexity and it better captures the flexibility of form-meaning associations found in human languages. Experiments are conducted using embodied agents playing situated language games. We present and discuss some interesting properties of the emergent dynamics exhibited by our model: first, the model seems to be able to handle some difficulties earlier models struggled with. Second, meanings of words can shift between being very specific (names) and general (e.g. thing). And third, not the model itself but the distribution of object properties in the world biases the specificity of words. 1

Agents in the process of bootstrapping a shared lexicon face immense uncertainty. The problem that an agent cannot point to meaning but only to objects, represents one of the core aspects of the problem. Even with a straightforward representation of meaning, such as a set of boolean features, the hypothesis space scales exponential in the number of primitive features. Furthermore, data suggests that human learners grasp aspects of many novel words after only a few exposures. We propose a model that can handle the exponential increase in uncertainty and allows scaling towards very large meaning spaces. The key novelty is that word learning or bootstrapping should not be viewed as a mapping task, in which a set of forms is to be mapped onto a set of (predefined) concepts. Instead we view word learning as a process in which the representation of meaning gradually shapes itself, while being usable in interpretation and production almost instantly. 1.

Abstract. In this paper, we explore how human-like social learning can be implemented in artificial life models. We focus on the social learning of both skills and language and we illustrate our considerations and design issues based on our developments in the NEW TIES project. We conclude that our assumptions regarding autonomy, embodiment and situatedness impose many limitations and, consequently require difficult design choices. 1

The papers collected in this special issue emerged from an international workshop on symbol grounding organised at the University of Plymouth on 3 and 4 July 2006 by the Distributed Language Group. Our goal was to extend the classical view of symbol grounding by recognising that language and cognitive dynamics are mutually constitutive. Specifically, we aimed to do so by bringing researchers who study human signalling together with others who focus on simulating intelligence and language. In the original call for papers, we set out these objectives as follows: Specifically, we wish to invite contributions viewing language and cognition as linking what goes on in the head with causal processes that are intersubjective, multimodal, affect-laden, and organised by historically rooted customs and artefacts. … The purpose of the workshop is not so much to present completed work as to find new ways of tackling a complex issue and to launch collaboration among participants to that end. … Since the workshop focuses on how symbol grounding can be reconsidered when language is viewed as a dynamical process rooted in both culture and biology, research

Date Approved: April 5, 2010for my wife, my family, and my friends, whose support made this possibleACKNOWLEDGEMENTS First, I would like to thank my advisor, Dr. Ron Arkin, whose support has made this dissertation possible. His critical reading of this work continually challenged me to improve upon it, and I think the result is much improved and much more accessible as a result. His vast knowledge of the field has been extremely helpful during the exploration process that has led to this topic. I would also like to thank the rest of the committee: Tucker Balch, Tom Collins, Ashok Goel, and Charles Isbell. The many helpful comments, especially during the formative stage of this work, helped me to better shape and define a substantive topic in robotics. Without my family, this dissertation would have never been possible. My parents were inspirational in their ability to start from nothing and achieve success despite the challenges. They continually pushed me to work hard and excel at whatever I do, and without this push I would have never reached this stage. My brother, who always

We introduce the novel problem of inter-robot transfer learning for perceptual classification of objects, where multiple heterogeneous robots communicate and transfer learned object models consisting of a fusion of multiple object properties. Unlike traditional transfer learning, there can be severe differences in the data distributions, resulting from differences in sensing, sensory processing, or even representations, that each robot uses to learn. Furthermore, only some properties may overlap between the two robots. We show that in such cases, the abstraction of raw sensory data into an intermediate representation can be used not only to aid learning, but also the transfer of knowledge. Further, we utilize statistical metrics, learned during an interactive process where the robots jointly explore the environment, to determine which underlying properties are shared between the robots. We demonstrate results in a visual classification task where objects are represented via a combination of properties derived from different modalities: color, texture, shape, and size. Using our methods, two heterogeneous robots utilizing different sensors and representations are able to successfully transfer support vector machine (SVM) classifiers among each other, resulting in speedups during learning.