This paper presents an implemented computational model of word acquisition which learns directly from raw multimodal sensory input. Set in an information theoretic framework, the model acquires a lexicon by finding and statistically modeling consistent cross-modal structure. The model has been implemented in a system using novel speech processing, computer vision, and machine learning algorithms. In evaluations the model successfully performed speech segmentation, word discovery and visual categorization from spontaneous infant-directed speech paired with video images of single objects. These results demonstrate the possibility of using state-of-the-art techniques from sensory pattern recognition and machine learning to implement cognitive models which can process raw sensor data without the need for human transcription or labeling.

The perceptual magnet effect is one of the earliest known language-specific phenomena arising in infant speech development. The effect is characterized by a warping of perceptual space near phonemic category centers. Previous explanations have been formulated within the theoretical framework of cognitive psychology. The model proposed in this paper builds on research from both psychology and neuroscience in working toward a more complete account of the effect. The model embodies two principal hypotheses supported by considerable experimental and theoretical research from the neuroscience literature: (1) sensory experience guides language-specific development of an auditory neural map, and (2) a population vector can predict psychological phenomena based on map cell activities. These hypotheses are realized in a selforganizing neural network model. The magnet effect arises in the model from language-specific nonuniformities in the distribution of map cell firing preferences. Numerical simulations verify that the model captures the known general characteristics of the magnet effect and provides accurate fits to specific

Abstract not found

This article describes a neural network model of speech motor skill acquisition and speech production that explains a wide range of data on variability, motor equivalence, coarticulation, and rate effects. Model parameters are learned during a babbling phase. To explain how infants learn language-specific variability limits, speech sound targets take the form of convex regions, rather than points, in orosensory coordinates. Reducing target size for better accuracy during slower speech leads to differential effects for vowels and consonants, as seen in experiments previously used as evidence for separate control processes for the 2 sound types. Anticipatory coarticulation arises when targets are reduced in size on the basis of context; this generalizes the well-known look-ahead model of coarticulation. Computer simulations verify the model's properties. The primary goal of the modeling work described in this article is to provide a coherent theoretical framework that provides explanations for a wide range of data concerning the articulator movements used by humans to produce speech sounds. This is carried out by formulating a model that transforms strings of phonemes into continuous articulator movements for

The magnification exponents ¯ occuring in adaptive map formation algorithms like Kohonen's self-organizing feature map deviate for the information theoretically optimal value ¯ = 1 as well as from the values which optimize, e.g., the mean square distortion error (¯ = 1=3 for one-dimensional maps). At the same time, models for categorical perception such as the "perceptual magnet" effect which are based on topographic maps require negative magnification exponents ¯ ! 0. We present an extension of the self-organizing feature map algorithm which utilizes adaptive local learning step sizes to actually control the magnification properties of the map. By change of a single parameter, maps with optimal information transfer, with various minimal reconstruction errors, or with an inverted magnification can be generated. Analytic results on this new algorithm are complemented by numerical simulations. 1. Introduction The representation of information in topographic maps is a common property of...

Abstract not found

seeing a toy car in it she says car; taking i::'..'. the ear out she says out; putting it on the floor she says down. In the world at large these little remarks do not command much attention. But to people i interested in how children learn to talk, the first steps into language raise fas- :7' "? elnating and difficult questions. In this chapter, we are concerned with the enal puzzle of where children's early word meanings come from. Are they ' introduced through language? Do they reflect concepts that arise spontane~ -. ously through infants' perceptual and cognitive development? Do language ":.: and cognition interact to produce early word meanings, and, if so, how? ",..., The idea that children learn how to structure meanings through exposure to language is' usually associated with Whorl (1956). Whorf stressed that ':: languages differ in the way they partition the world, and he proposed that in learning the semantic categories of their language, children also acquire a 'world vie

Psychophysical phenomena such as categorical perception and the perceptual magnet effect indicate that our auditory perceptual spaces are warped for some stimuli. This paper investigates the effects of two different kinds of training on auditory perceptual space. It is first shown that categorization training using non-speech stimuli, in which subjects learn to identify stimuli within a particular frequency range as members of the same category, can lead to a decrease in sensitivity to stimuli in that category. This phenomenon is an example of acquired similarity and apparently has not been previously demonstrated for a category -relevant dimension. Discrimination training with the same set of stimuli was shown to have the opposite effect: subjects became more sensitive to differences in the stimuli presented during training. Further experiments investigated some of the conditions that are necessary to generate the acquired similarity found in the first experiment. The results of these...

Properties of the physical world have shaped human evolutionary design and given rise to physically grounded mental representations. These grounded representations provide the foundation for higher level cognitive processes including language. Most natural language processing machines to date lack grounding. This paper advocates the creation of physically grounded language learning machines as a path toward scalable systems which can conceptualize and communicate about the world in human-like ways. As steps in this direction, two experimental language acquisition systems are presented.

Abstract not found