1.1. Conceptual systems 621 1.2. Semantic memory 621

The suggestion that the body surface might be used as an additional means of presenting information to human-machine operators has been around in the literature for nearly 50 years. Although recent technological advances have made the possibility of using the body as a receptive surface much more realistic, the fundamental limitations on the human information processing of tactile stimuli presented across the body surface are, however, still largely unknown. This literature review provides an overview of studies that have attempted to use vibrotactile interfaces to convey information to human operators. The importance of investigating any possible central cognitive limitations (i.e., rather than the peripheral limitations, such as related to sensory masking, that were typically addressed in earlier research) on tactile processing for the most effective design of body interfaces is highlighted. The applicability of the constraints emerging from studies of tactile processing under conditions of unisensory (i.e., purely tactile) stimulus presentation, to more ecologically valid conditions of multisensory stimulation, is also discussed. Finally, the results obtained from recent studies of tactile information processing under conditions of multisensory stimulation are described, and their implications for haptic/tactile interface design elucidated.

According to perceptual symbol systems (Barsalou, 1999), sensory-motor simulations underlie the representation of concepts. It follows that sensory-motor phenomena should arise in conceptual processing. Previous studies have shown that switching from one modality to another during perceptual processing incurs a processing cost. If perceptual simulation underlies conceptual processing, then verifying the properties of concepts should exhibit a switching cost as well. For example, verifying a property in the auditory modality (e.g., BLENDER-loud) should be slower after verifying a property in a different modality (e.g., CRANBERRIES-tart) than in the same modality (e.g., LEAVES-rustling). Only words were presented to subjects, and there were no instructions to use imagery. Nevertheless switching modalities incurred a cost, analogous to switching modalities in perception. A second experiment showed that this effect was not due to associative priming between properties in the same modality. These results support the hypothesis that perceptual simulation underlies conceptual processing. Modern psychology relies heavily on the digital computer as a metaphor for human cognition (e.g., Fodor, 1975; Pylyshyn, 1984). According to this view, the software of the mind can be distinguished from the hardware of the body, with mental representations being amodal redescriptions of sensory-motor experience. Increasingly, however, researchers argue that this approach is fundamentally wrong, suggesting instead that interactions between sensory-motor systems and the physical world underlie cognition. For example, Barsalou's (1999) theory of perceptual symbol systems proposes that conceptual knowledge is grounded in sensory-motor systems. To represent a concept, neural systems partially run a...

According to perceptual symbol systems, sensorimotor simulations underlie the representation of concepts. It follows that sensorimotor phenomena should arise in conceptual processing. Previous studies have shown that switching from one modality to another during perceptual processing incurs a processing cost. If perceptual simulation underlies conceptual processing, then verifying the properties of concepts should exhibit a switching cost as well. For example, verifying a property in the auditory modality (e.g., BLENDER-loud) should be slower after verifying a property in a different modality (e.g., CRANBERRIES-tart) than after verifying a property in the same modality (e.g., LEAVES-rustling). Only words were presented to subjects, and there were no instructions to use imagery. Nevertheless, switching modalities incurred a cost, analogous to the cost of switching modalities in perception. A second experiment showed that this effect was not due to associative priming between properties in the same modality. These results support the hypothesis that perceptual simulation underlies conceptual processing.

Abstract not found

According to the Perceptual Symbols Theory of cognition (Barsalou, 1999), modality-specific simulations underlie the representation of concepts. A strong prediction of this view is that perceptual processing affects conceptual processing. In this study, participants performed a perceptual detection task and a conceptual property-verification task in alternation. Responses on the property-verification task were slower for those trials that were preceded by a perceptual trial in a different modality than for those that were preceded by a perceptual trial in the same modality. This finding of a modality-switch effect across perceptual processing and conceptual processing supports the hypothesis that perceptual and conceptual representations are partially based on the same systems.

The potential use of non-visual warning signals to present spatial information to car drivers has been successfully demonstrated in several recent studies (Ho & Spence, submitted, in preparation; Ho, Tan, & Spence, submitted). Among the three types of spatial warning signals investigated (namely auditory, visual, and vibrotactile), spatial vibrotactile cues were found to be particularly effective in directing a driver’s visual spatial attention to potentially dangerous events on the road. We conducted the present study in order to examine the factors governing the relative effectiveness of auditory, visual, and vibrotactile warning signals. The speeded discrimination of warning signals presented in the various different modalities was investigated in order to explore whether the differences found in our previous research were a result of the relative speed with which people can detect warning signals presented in a given modality, or whether they were attributable to differences in the efficacy with which people can relate the warning signal to the subsequent visually-specified target driving events. 1

Previous research has shown that the presentation of spatially predictive auditory and vibrotactile warning signals can facilitate driver responses to driving events seen through the windscreen or rearview mirror. The present study investigated whether this facilitation reflects the priming of the appropriate response (i.e. braking vs. accelerating) or an attentional cuing effect (i.e. a perceptual benefit that facilitates subsequent behavioural responding). In the experiments reported here, participants had to discriminate the colour of a number plate (red vs. blue) following the presentation of either spatially predictive vibrotactile (experiment 1) or auditory (experiment 2) warning signals that indicated the likely location (front or back) of the visual target, while simultaneously performing a highly attention-demanding rapid serial visual presentation task. Numberplate discrimination performance was facilitated following the presentation of valid auditory cues, but not following the presentation of equally informative vibrotactile cues. The use of an orthogonal spatial cuing design enabled with us to rule out of a potential response priming account of these data. The results suggest that whilst directional congruency between a warning signal and a target event may be sufficient to facilitate performance due to the priming of the appropriate response, attentional facilitation effects may also require the co-location of the cue and target within the same functional region of space.

Theories of embodied cognition hold that higher cognitive processes operate on perceptual symbols and that concept use involves partial reactivations of the sensory-motor states that occur during experience with the world. On this view, the processing of emotion knowledge involves a (partial) reexperience of an emotion, but only when access to the sensory basis of emotion knowledge is required by the task. In 2 experiments, participants judged emotional and neutral concepts corresponding to concrete objects (Experiment 1) and abstract states (Experiment 2) while facial electromyographic activity was recorded from the cheek, brow, eye, and nose regions. Results of both studies show embodiment of specific emotions in an emotion-focused but not a perceptual-focused processing task on the same words. A follow up in Experiment 3, which blocked selective facial expressions, suggests a causal, rather than simply a correlational, role for embodiment in emotion word processing. Experiment 4, using a property generation task, provided support for the conclusion that emotions embodied in conceptual tasks are context-dependent situated simulations rather than associated emotional reactions. Implications for theories of embodied simulation and for emotion theories are discussed.

Previous research has shown that the use of multimodal signals can lead to faster and more accurate responses compared to purely unimodal displays. However, in most cases response facilitation only occurs when the signals are presented in roughly the same spatial location. This would suggest a severe restriction on interface designers: to use multimodal displays effectively all signals must be presented from the same location on the display. We previously reported evidence that the use of haptic cues may provide a solution to this problem as haptic cues presented to a user’s back canbeusedtoredirectvisualattentiontolocations on a screen in front of the user [1]. In the present experiment we used a visual change detection task to investigate whether (i) this type of visual-haptic interaction is robust at low cue validity rates and (ii) similar effects occur for auditory cues. Valid haptic cues resulted in significantly faster change detection times even when they accurately indicated the location of the change on only 20 % of the trials. Auditory cues had a much smaller effect on detection times at the high validity rate (80%) than haptic cues and did not significantly improve performance at the 20 % validity rate. These results suggest that the use haptic attentional cues may be particularly effective in environments in which information cannot be presented in the same spatial location. 1.