How can we build robots that engage in fluid spoken conversations with people, moving beyond canned responses to words and towards actually understanding? As a step towards addressing this question, we introduce a robotic architecture that provides a basis for grounding word meanings. The architecture provides perceptual, procedural, and affordance representations for grounding words. A perceptuallycoupled on-line simulator enables sensorymotor representations that can shift points of view. Held together, we show that this architecture provides a rich set of data structures and procedures that provide the foundations for grounding the meaning of certain classes of words.

The audio bitstream in music encodes a high amount of statistical, acoustic, emotional and cultural information. But music also has an important linguistic accessory; most musical artists are described in great detail in record reviews, fan sites and news items. We highlight current and ongoing research into extracting relevant features from audio and simultaneously learning language features linked to the music. We show results in a "query-by-description" task in which we learn the perceptual meaning of automatically-discovered single-term descriptive components, as well as a method of automatically uncovering `semantically attached' terms (terms that have perceptual grounding.) We then show recent work in `semantic basis functions' -- parameter spaces of description (such as fast ... slow or male ... female) that encode the highest descriptive variance in a semantic space.

Abstract not found

TWIG (“Transportable Word Intension Generator”) is a system that allows a robot to learn compositional meanings for new words that are grounded in its sensory capabilities. The system is novel in its use of logical semantics to infer which entities in the environment are the referents (extensions) of unfamiliar words; its ability to learn the meanings of deictic (“I, ” “this”) pronouns in a real sensory environment; its use of decision trees to implicitly contrast new word definitions with existing ones, thereby creating more complex definitions than if each word were treated as a separate learning problem; and its ability to use words learned in an unsupervised manner in complete grammatical sentences for production, comprehension, or referent inference. In an experiment with a physically embodied robot, TWIG learns grounded meanings for the words “I ” and “you, ” learns that “this ” and “that ” refer to objects of varying proximity, that “he ” is someone talked about in the third person, and that “above ” and “below ” refer to height differences between objects. Follow-up experiments demonstrate the system’s ability to learn different conjugations of “to be”; show that removing either the extension inference or implicit contrast components of the system results in worse definitions; and demonstrate how decision trees can be used to model shifts in meaning based on context in the case of color words.

Computational models have played a central role in the debate over language learnability. This article discusses how they have been used in different “stances”, from generative views to more recently introduced explanatory frameworks based on embodiment, cognitive development and cultural evolution. By digging into the details of certain specific models, we show how they organize, transform and rephrase defining questions about what makes language learning possible for children. Finally, we present a tentative synthesis to recast the debate using the notion of learning bias.

Drawing on data from linguistics, developmental psychology and the neurosciences, we present a computational theory of the acquisition of early grammar by infants. Based on the view that language is a mapping between form and meaning, we propose that a theory of language acquisition must be tightly integrated with a theory of the infant's prelinguistic representations. Namely, the infant's task is to learn how to map the linguistic form in the input to her representations of the corresponding scenes. We have developed a theory of prelinguistic cognition based on i) what is currently known about the architecture of the brain, and ii) the representational requirements for successful (sensorimotor) behavior in the world. We show how such prelinguistic sensorimotor representations can provide the basis for the acquisition of early grammatical forms, and thereby ground language in the world. Importantly, this is true not only at the lexical level, but also at the grammatical level.

The aim of the present volume is to enrich human language dimensions by seeking to understand how the use of language may be situated with respect to other systems for action and perception. There is strong evidence that higher human cognitive functions, such as imitation and language, emerged from or co-evolved with the ability for compositionality

Abstract — The authors present a method by which a robot can learn the meanings of words from unlabeled correct examples in context. The “word trees ” method consists of reconstructing the speaker’s decision process in choosing a word. The facts about an object and its relation to other objects that maximally reduce the uncertainty (entropy) of word choice become the decision nodes of this tree. The conjunction of the choices leading to a word becomes its logical definition. Definitions thereby become only as complex as is necessary to distinguish words in the vocabulary, making the method appear to follow a heuristic that developmental psychologists call the “Principle of Contrast. ” Combined with a method for inferring word type and reference, the method produces semantics complete enough to produce or understand full sentences. The method was implemented on a robot with visual, auditory, and positional sensors, and succeeded in learning the differences between “I, ” “you, ” “he, ” “this, ” “that, ” “above,” “below, ” and “near.” I.

We propose a theoretical framework for modeling communication between agents that have different conceptual models of their current context. We describe how the emergence of subjective models of the world can be simulated and what the role of language and communication in that process is. We consider, in particular, the role of unsupervised learning in the formation of agents ' conceptual models, the relative subjectivity of these models, and the communication and learning processes that lead into intersubjective sharing of concepts. We also discuss some implications of the subjectivity of conceptual learning in the area of economics. 1

The Neural Theory of Language project aims to build structured connectionist models of language and cognition consistent with constraints from all domains and at all levels. These constraints include recent experimental evidence that details of neural computation and brain architecture play a crucial role in language processing. We focus in this paper on the computational level and explore the role of embodied representations and simulative inference in language understanding. 1.