Recent neurological studies indicate that the role of emotion in human cognition is essential; emotions are not a luxury. Instead, emotions play a critical role in rational decision-making, in perception, in human interaction, and in human intelligence. These facts, combined with abilities computers are acquiring in expressing and recognizing affect, open new areas for research. This paper defines key issues in "affective computing," computing that relates to, arises from, or deliberately influences emotions. New models are suggested for computer recognition of human emotion, and both theoretical and practical applications are described for learning, human-computer interaction, perceptual information retrieval, creative arts and entertainment, human health, and machine intelligence. Significant potential advances in emotion and cognition theory hinge on the development of affective computing, especially in the form of wearable computers. This paper establishes challenges and future directions for this emerging field.

We investigated grapheme--colour synaesthesia and found that: (1) The induced colours led to perceptual grouping and pop-out, (2) a grapheme rendered invisible through `crowding' or lateral masking induced synaesthetic colours --- a form of blindsight --- and (3) peripherally presented graphemes did not induce colours even when they were clearly visible. Taken collectively, these and other experiments prove conclusively that synaesthesia is a genuine perceptual phenomenon, not an effect based on memory associations from childhood or on vague metaphorical speech. We identify different subtypes of number--colour synaesthesia and propose that they are caused by hyperconnectivity between colour and number areas at different stages in processing; lower synaesthetes may have cross-wiring (or cross-activation) within the fusiform gyrus, whereas higher synaesthetes may have cross-activation in the angular gyrus. This hyperconnectivity might be caused by a genetic mutation that causes defective pruning of connections between brain maps. The mutation may further be expressed selectively (due to transcription factors) in the fusiform or angular gyri, and this may explain the existence of different forms of synaesthesia. If expressed very diffusely, there may be extensive cross-wiring between brain regions that represent abstract concepts, which would explain the link between creativity, metaphor and synaesthesia (and the higher incidence of synaesthesia among artists and poets). Also, hyperconnectivity between the sensory cortex and amygdala would explain the heightened aversion synaesthetes experience when seeing numbers printed in the `wrong' colour. Lastly, kindling (induced hyperconnectivity in the temporal lobes of temporal lobe epilepsy [TLE] patients) may explain the purp...

The history of abstract animation and light performance points towards an aesthetic of temporal abstraction which digital computer graphics can ideally explore. Computer graphics has leapt forward to embrace three-dimensional texture mapped imagery, but stepped over the broad aesthetic terrain of two-dimensional interactive dynamic abstraction. Several experiments in using pure human movement as the interface to dynamic abstract systems are presented with the goal of creating phenomenological interfaces that engage the unconscious mind directly. These applications are visual instruments that allow immediate understanding of a dynamic system, but point towards infinite challenges in their mastery as any good artistic medium. The lessons from these experim...

This paper will attempt to define a new area of research: affective communication through the use of affective objects. An affective object may be defined as any physical object which has the ability to sense emotional data from a person, map that information to an abstract form of expression and communicate that information expressively, either back to the subject herself or to another person. This paper will also propose a theory of how to utilize existing knowledge (i.e. how humans interpret the expressive qualities of modalities such as color, sound, and light) to inform the invention process. 1

In this paper, we will present an approach to design a more natural user interface without taking resort to special haptic input/output devices. Tactile sensations like stickiness, touch, or mass can be evoked by applying tiny displacements upon cursor movements. Our active cursor method exploits the domination of the visual over the haptic domain. We will show that interactive animations can be used to simulate the functioning of force-feedback devices. A demo is online at

Abstract. Our engagement with consumer products diminishes gradually over the last decades, which causes considerable usability problems. To dissolve these problems, the designer's emphasis should shift from creating beautiful products in appearance to beautiful interactions with products. Consequently, the designer needs new tools like gestural sketching. To develop a gestural design tool, we tested the suitability of gestures to capture expressive ideas and the capability of outsiders to recognise this expression. Scents were used to make this expression measurable. Twentytwo creators made four dynamic sculptures expressing these scents. Half of those sculptures were made through gesturing and half through traditional sketching. Subjects were asked to match the scents and the sculptures. Results show that there is no significant difference between sketching and gesturing. Dependent on the scent, an interpreter was able to capture the expression when looking at the gestures. These findings support the potential of a gestural design tool.

A design for an implemented, prototype wearable artificial sensory system is presented, which uses data sonification to compensate for normal limitations in the human visual system. The system gives insight into the complete visible-light spectra from objects being seen by the user. Long-term wear and consequent training might lead to identification of various visually-indistinguishable materials based on the sounds of their spectra. A detailed system design and results of user testing are presented, and many possible extensions to both the sonification and the sensor package are discussed. 1.

Abstract not found

We present the results of user studies that were performed on sighted people to test their ability to detect simple shapes with SoundView. SoundView is an experimental vision substitution system for the blind. Visual images are mapped onto a virtual surface with a fine-grained color dependent roughness texture. The user explores an image by moving a pointer device over the image which creates sounds. The current prototype uses a Wacom graphics tablet as a pointer device. The pointer acts like a virtual gramophone needle, and the sound produced depends on the motion as well as on the color of the area explored. An extension of SoundView also allows haptic feedback and we have compared the performance of users using auditory and/or haptic feedback.

Humanistic computing is proposed as a new signal processing framework in which the processing apparatus is inextricably intertwined with the natural capabilities of our human body and mind. Rather than trying to emulate human intelligence, humanistic computing recognizes that the human brain is perhaps the best neural network of its kind, and that there are many new signal processing applications (within the domain of personal technologies) that can make use of this excellent but often overlooked processor. The emphasis of this paper is on personal imaging applications of humanistic computing, to take a first step toward an intelligent wearable camera system that can allow us to effortlessly capture our day-to-day experiences, help us remember and see better, provide us with personal safety through crime reduction, and facilitate new forms of communication through collective connected humanistic computing. The author’s wearable signal processing hardware, which began as a cumbersome backpackbased photographic apparatus of the 1970’s and evolved into a clothing-based apparatus in the early 1980’s, currently provides the computational power of a UNIX workstation concealed within ordinary-looking eyeglasses and clothing. Thus it may be worn continuously during all facets of ordinary day-to-day living, so that, through long-term adaptation, it begins to function as a true extension of the mind and body.