We report a new phenomenon associated with language comprehension: the action–sentence compatibility effect (ACE). Participants judged whether sentences were sensible by making a response that required moving toward or away from their bodies. When a sentence implied action in one direction (e.g., “Close the drawer ” implies action away from the body), the participants had difficulty making a sensibility judgment requiring a response in the opposite direction. The ACE was demonstrated for three sentences types: imperative sentences, sentences describing the transfer of concrete objects, and sentences describing the transfer of abstract entities, such as “Liz told you the story. ” These data are inconsistent with theories of language comprehension in which meaning is represented as a set of relations among nodes. Instead, the data support an embodied theory of meaning that relates the meaning of sentences to human action. How language conveys meaning remains an open question. The dominant approach is to treat language as a symbol manipulation system: Language conveys meaning by using abstract, amodal, and arbitrary symbols (i.e., words) combined by syntactic rules (e.g., Burgess & Lund, 1997; Chomsky, 1980; Fodor, 2000; Kintsch, 1988; Pinker, 1994). Words are abstract in that the same word, such as “chair, ” is used for big chairs and little chairs, words are amodal in that the same word is used when chairs are spoken about or written about, and words are arbitrarily related to their referents in that the phonemic and orthographic characteristics of a word bear no relationship to the physical or functional characteristics of the word’s referent. An alternative view is that linguistic meaning is

The emerging viewpoint of embodied cognition holds that cognitive processes are deeply rooted in the body’s interactions with the world. This position actually houses a number of distinct claims, some of which are more controversial than others. This paper distinguishes and evaluates the following six claims: 1) cognition is situated; 2) cognition is time-pressured; 3) we off-load cognitive work onto the environment; 4) the environment is part of the cognitive system; 5) cognition is for action; 6) off-line cognition is body-based. Of these, the first three and the fifth appear to be at least partially true, and their usefulness is best evaluated in terms of the range of their applicability. The fourth claim, I argue, is deeply problematic. The sixth claim has received the least attention in the literature on embodied cognition, but it may in fact be the best documented and most powerful of the six claims.

A theoretical framework for grounding language is introduced that provides a computational path from sensing and motor action to words and speech acts. The approach combines concepts from semiotics and schema theory to develop a holistic approach to linguistic meaning. Schemas serve as structured beliefs that are grounded in an agent’s physical environment through a causal-predictive cycle of action and perception. Words and basic speech acts are interpreted in terms of grounded schemas. The framework reflects lessons learned from implementations of several language processing robots. It provides a basis for the analysis and design of situated, multimodal communication systems that straddle symbolic and non-symbolic realms.

model meaning as the relations among abstract symbols that are arbitrarily related to what they signify. These symbols are ungrounded in that they are not tied to perceptual experience or action. Because the symbols are ungrounded, they cannot, in principle, capture the meaning of novel situations. In contrast, participants in three experiments found it trivially easy to discriminate between descriptions of sensible novel situations (e.g., using a newspaper to protect one’s face from the wind) and nonsense novel situations (e.g., using a matchbook to protect one’s face from the wind). These results support the Indexical Hypothesis that the meaning of a sentence is constructed by (a) indexing words and phrases to real objects or perceptual, analog symbols; (b) deriving affordances from the objects and symbols; and (c) meshing the affordances under the guidance of syntax. © 2000 Academic Press Key Words: meaning; language; embodiment; computational models; Latent Semantic Analysis; Hyperspace Analogue to Language. Meaning is the most important problem in cognitive psychology. Meaning controls memory and perception. Meaning is the goal of communication. Meaning underlies social activities and culture: To a great degree, what distinguishes human cultures are the meanings they give to natural phenomena, artifacts, and human relations. Yet, rather than being a hotbed of theoretical and empirical investigation, meaning in cognitive psychology has been coopted by a particular approach: Meaning arises

To build robots that engage in fluid face-to-face spoken conversations with people, robots must have ways to connect what they say to what they see. A critical aspect of how language connects to vision is that language encodes points of view. The meaning of my left and your left differs due to an implied shift of visual perspective. The connection of language to vision also relies on object permanence. We can talk about things that are not in view. For a robot to participate in situated spoken dialog, it must have the capacity to imagine shifts of perspective, and it must maintain object permanence. We present a set of representations and procedures that enable a robotic manipulator to maintain a “mental model” of its physical environment by coupling active vision to physical simulation. Within this model, “imagined” views can be generated from arbitrary perspectives, providing the basis for situated language comprehension and production. An initial application of mental imagery for spatial language understanding for an interactive robot is described.

The authors present a theory of how relational inference and generalization can be accomplished within a cognitive architecture that is psychologically and neurally realistic. Their proposal is a form of symbolic connectionism: a connectionist system based on distributed representations of concept meanings, using temporal synchrony to bind fillers and roles into relational structures. The authors present a specific instantiation of their theory in the form of a computer simulation model, Learning and Inference with Schemas and Analogies (LISA). By using a kind of self-supervised learning, LISA can make specific inferences and form new relational generalizations and can hence acquire new schemas by induction from examples. The authors demonstrate the sufficiency of the model by using it to simulate a body of empirical phenomena concerning analogical inference and relational generalization. A fundamental aspect of human intelligence is the ability to form and manipulate relational representations. Examples of relational thinking include the ability to appreciate analogies between seemingly different objects or events (Gentner, 1983; Holyoak & Thagard, 1995), the ability to apply abstract rules in novel situations (e.g., Smith, Langston, & Nisbett, 1992), the ability to understand and learn language (e.g., Kim, Pinker, Prince, & Prasada, 1991), and even the ability to appreciate perceptual similarities

Abstract not found

Three theories currently compete to explain the conceptual deficits that result from brain damage: sensory-functional theory, domain-specific theory, and conceptual structure theory. We argue that all three theories capture important aspects of conceptual deficits, and offer different insights into their origins. Conceptual topography theory (CTT) integrates these insights, beginning with A. R. Damasio’s (1989) convergence zone theory and elaborating it with the similarity-in-topography (SIT) principle. According to CTT, feature maps in sensory-motor systems represent the features of a category’s exemplars. A hierarchical system of convergence zones then conjoins these features to form both property and category representations. According to the SIT principle, the proximity of two conjunctive neurons in a convergence zone increases with the similarity of the features they conjoin. As a result, conjunctive neurons become topographically organised into local regions that represent properties and categories. Depending on the level and location of a lesion in this system, a wide variety of deficits is possible. Consistent with the literature, these deficits range from the loss of a single category to the loss of multiple categories that share sensory-motor properties.

A spoken language generation system has been developed that learns to describe objects in computer-generated visual scenes. The system is trained by a `show-and-tell' procedure in which visual scenes are paired with natural language descriptions. Learning algorithms acquire probabilistic structures which encode the visual semantics of phrase structure, word classes, and individual words. Using these structures, a planning algorithm integrates syntactic, semantic, and contextual constraints to generate natural and unambiguous descriptions of objects in novel scenes.

This article presents a formal theory of robot perception as a form of abduction. The theory pins down the process whereby low-level sensor data is transformed into a symbolic representation of the external world, drawing together aspects such as incompleteness, top-down information flow, active perception, attention, and sensor fusion in a unifying framework. In addition, a number of themes are identified that are common to both the engineer concerned with developing a rigorous theory of perception, such as the one on offer here, and the philosopher of mind who is exercised by questions relating to mental representation and intentionality.