Facial expressions provide an important behavioral measure for the study of emotion, cognitive processes, and social interaction. The Facial Action Coding System ~Ekman & Friesen, 1978! is an objective method for quantifying facial movement in terms of component actions. We applied computer image analysis to the problem of automatically detecting facial actions in sequences of images. Three approaches were compared: holistic spatial analysis, explicit measurement of features such as wrinkles, and estimation of motion flow fields. The three methods were combined in a hybrid system that classified six upper facial actions with 91 % accuracy. The hybrid system outperformed human nonexperts on this task and performed as well as highly trained experts. An automated system would make facial expression measurement more widely accessible as a research tool in behavioral science and investigations of the neural substrates of emotion.

Neurophysiological evidence is described, showing that some neurons in the macaque temporal cortical visual areas have responses that are invariant with respect to the position, size and view of faces and objects, and that these neurons show rapid processing and rapid learning. A theory is then described of how such invariant representations may be produced in a hierarchically organized set of visual cortical areas with convergent connectivity. The theory proposes that neurons in these visual areas use a modified Hebb synaptic modification rule with a short-term memory trace to capture whatever can be captured at each stage that is invariant about objects as the object changes in retinal position, size, rotation and view. Simulations are then described which explore the operation of the architecture. The simulations show that such a processing system can build invariant representations of objects.

Increased knowledge about the ways people recognize each other may help to guide efforts to develop practical automatic face-recognition systems.

We used functional magnetic resonance imaging to study the response properties of the human fusiform face area (FFA: Kanwisher, McDermott, & Chun, 1997) to a variety of face-like stimuli in order to clarify the functional role of this region. FFA responses were found to be (1) equally strong for cat, cartoon and human faces despite very different image properties, (2) equally strong for entire human faces and faces with eyes occluded but weaker for eyes shown alone, (3) equal for front and profile views of human heads, but declining in strength as faces rotated away from view, and (4) weakest for nonface objects and houses. These results indicate that generalisation of the FFA response across very different face types cannot be explained in terms of a specific response to a salient facial feature such as the eyes or a more general response to heads. Instead, the FFA appears to be optimally tuned to the broad category of faces.

Human episodic memory provides a seemingly unlimited storage for everyday experiences, and a retrieval system that allows us to access the experiences with partial activation of their components. The system is believed to consist of a fast, temporary storage in the hippocampus, and a slow, longterm storage within the neocortex. This paper presents a neural network model of the hippocampal episodic memory inspired by Damasio's idea of Convergence Zones. The model consists of a layer of perceptual feature maps and a binding layer. A perceptual feature pattern is coarse coded in the binding layer, and stored on the weights between layers. A partial activation of the stored features activates the binding pattern, which in turn reactivates the entire stored pattern. For many configurations of the model, a theoretical lower bound for the memory capacity can be derived, and it can be an order of magnitude or higher than the number of all units in the model, and several orders of magnitude higher than the number of binding-layer units. Computational simulations further indicate that the average capacity is an order of magnitude larger than the theoretical lower bound, and making the connectivity between layers sparser causes an even further increase in capacity. Simulations also show that if more descriptive binding patterns are used, the errors tend to be more plausible (patterns are confused with other similar patterns), with a slight cost in capacity. The convergence-zone episodic memory therefore accounts for the immediate storage and associative retrieval capability and large capacity of the hippocampal memory, and shows why the memory encoding areas can be much smaller than the perceptual maps, consist of rather coarse computational units, and be only sparsely connected t...

DISLEX is an artificial neural network model of the mental lexicon. It was built to test computationally whether the lexicon could consist of separate feature maps for the different lexical modalities and the lexical semantics, connected with ordered pathways. In the model, the orthographic, phonological, and semantic feature maps and the associations between them are formed in an unsupervised process, based on cooccurrence of the lexical symbol and its meaning. After the model is organized, various damage to the lexical system can be simulated, resulting in dyslexic and category-specific aphasic impairments similar to those observed in human patients. 1 Introduction The human lexical system is believed to be highly modular, consisting of a central semantic component and separate symbol memories for the different input and output modalities (Caramazza 1988; McCarthy and Warrington 1990). Such an architecture is intuitively compelling since the modalities give rise to different repres...

Recognizing emotion from facial expressions draws on diverse psychological processes implemented in a large array of neural structures. Studies using evoked potentials, lesions, and functional imaging have begun to elucidate some of the mechanisms. Early perceptual processing of faces draws on cortices in occipital and temporal lobes that construct detailed representations from the configuration of facial features. Subsequent recognition requires a set of structures, including amygdala and orbitofrontal cortex, that links perceptual representations of the face to the generation of knowledge about the emotion signaled, a complex set of mechanisms using multiple strategies. Although recent studies have provided a wealth of detail regarding these mechanisms in the adult human brain, investigations are also being extended to nonhuman primates, to infants, and to patients with psychiatric disorders.

A large amount of data is now available about the pattern of connections between brain regions. Computational methods are increasingly relevant for uncovering structure in such datasets. There has been recent interest in the use of Nonmetric Multidimensional Scaling (NMDS) for such analysis (Young, 1992, 1993; Scannell & Young, 1993). NMDS produces a spatial representation of the "dissimilarities" between a number of entities. Normally, it is applied to data matrices containing a large number of levels of dissimilarity, whereas for connectivity data there is a very small number. We address the suitability of NMDS for this case. Systematic numerical studies are presented to evaluate the ability of this method to reconstruct known geometrical configurations from dissimilarity data possessing few levels. In this case there is a strong bias for NMDS to produce annular configurations, whether or not such structure exists in the original data. Using a connectivity dataset derived from the pr...

frontal lobes

Abstract. Using psychophysics we investigated to what extent human face recognition relies on local information in parts (featural information) and on their spatial relations (configural information). This is particularly relevant for biologically motivated computer vision since recent approaches have started considering such featural information. In Experiment 1 we showed that previously learnt faces could be recognized by human subjects when they were scrambled into constituent parts. This result clearly indicates a role of featural information. Then we determined the blur level that made the scrambled part versions impossible to recognize. This blur level was applied to whole faces in order to create configural versions that by definition do not contain featural information. We showed that configural versions of previously learnt faces could be recognized reliably. In Experiment 2 we replicated these results for familiar face recognition. Both Experiments provide evidence in favor of the view that recognition of familiar and unfamiliar faces relies on featural and configural information. Furthermore, the balance between the two does not differ for familiar and unfamiliar faces. We propose an integrative model of familiar and unfamiliar face recognition and discuss implications for biologically motivated computer vision algorithms for face recognition.