TWIG (“Transportable Word Intension Generator”) is a system that allows a robot to learn compositional meanings for new words that are grounded in its sensory capabilities. The system is novel in its use of logical semantics to infer which entities in the environment are the referents (extensions) of unfamiliar words; its ability to learn the meanings of deictic (“I, ” “this”) pronouns in a real sensory environment; its use of decision trees to implicitly contrast new word definitions with existing ones, thereby creating more complex definitions than if each word were treated as a separate learning problem; and its ability to use words learned in an unsupervised manner in complete grammatical sentences for production, comprehension, or referent inference. In an experiment with a physically embodied robot, TWIG learns grounded meanings for the words “I ” and “you, ” learns that “this ” and “that ” refer to objects of varying proximity, that “he ” is someone talked about in the third person, and that “above ” and “below ” refer to height differences between objects. Follow-up experiments demonstrate the system’s ability to learn different conjugations of “to be”; show that removing either the extension inference or implicit contrast components of the system results in worse definitions; and demonstrate how decision trees can be used to model shifts in meaning based on context in the case of color words.

Summarising georeferenced (can be identified according to it’s location) data in natural language is challenging because it requires linking events describing its nongeographic attributes to their underlying geography. This mapping is not straightforward as often the only explicit geographic information such data contains is latitude and longitude. In this paper we present an approach to generating textual summaries of georeferenced data based on spatial reference frames. This approach has been implemented in a data-to-text system we have deployed in the weather forecasting domain. 1

We describe our initial investigations into generating textual summaries of spatiotemporal data with the help of a prototype Natural Language Generation (NLG) system that produces pollen forecasts for Scotland. 1

We introduce an approach for physically-grounded natural language interpretation by robots which reacts appropriately to unanticipated physical changes in the environment and dynamically assimilates new information pertinent to ongoing tasks. At the core of the approach is a model of object schemas that enables a robot to encode beliefs about physical objects in its environment using collections of coupled processes responsible for sensorimotor interaction. These interaction processes run concurrently in order to ensure responsiveness to the environment, while coordinating sensorimotor expectations, action planning, and language use. The model has been implemented on a robot that manipulates objects on a tabletop in response to verbal input. The implementation responds to verbal requests such as “Group the green block and the red apple, ” while adapting in real-time to unexpected physical collisions and taking opportunistic advantage of any new information it may receive through perceptual and linguistic channels.

We introduce Wubble World, a virtual environment for learning situated language. In Wubble World children create avatars, called “wubbles, ” which can interact with other children’s avatars through free-form spontaneous play or structured language games. Wubbles can also learn language from direct interaction with children, since the system uses principles from developmental psychology to restrict the complexity of this learning task: a shared attention model that includes deictic pointing, and a concept acquisition system that allows for rapid learning of new words from a limited number of exposures. Since we have complete knowledge of the state

We present a general framework and learning algorithm for the task of concept labeling: each word in a given sentence has to be tagged with the unique physical entity (e.g. person, object or location) or abstract concept it refers to. Our method allows both world knowledge and linguistic information to be used during learning and prediction. We show experimentally that we can handle natural language and learn to use world knowledge to resolve ambiguities in language, such as word senses or coreference, without the use of hand-crafted rules or features. 1

www.elsevier.com/locate/artint Robotic vocabulary building using extension inference

This heretical article suggests that while embodiment was key to evolving human culture, and clearly affects our thinking and word choice now (as do many things in our environment), our culture may have evolved to such a point that a purely memetic AI beast could pass the Turing test. Though making something just like a human would clearly require both embodiment and memetics, if we were forced to choose one or the other, memetics might actually be easier. This short paper argues this point, and discusses what it would take to move beyond current semantic priming results to a human-like agent. 1

Topics of the Workshop Competence in spatial language modeling is a cardinal issue in disciplines including Cognitive Psychology, Computational Linguistics, and Computer Science. Within Cognitive Psychology, the relation of spatial language to models of spatial representation and reasoning is considered essential to the development of more complete models of psycholinguistic and cognitive linguistic theories [1]. Meanwhile, within Computer Science and Computational Linguistics and Engineering, the development of a wide class of so-called situated systems such as robotics, virtual characters, and Geographic Information Systems is heavily dependent on the existence of adequate models of spatial language use [2]. Achieving competence in spatial language requires that

We present a study on how people use size modifiers when referring to visible objects. We find strong evidence that the selection of modifiers like tall, thin, and big is brought about by several interacting factors, including how a target object’s physical dimensions differ from another object of the same type, and the relationship between the target object’s individual dimensions. Findings from this study are used to inform the design of a referring expression generation algorithm capable of referring to objects naturally, providing a further link between visual cues and corresponding linguistic forms.