Despite intense and wide-spread research in both virtual environments (VEs) and usability, the exciting new technology of VEs has not yet been closely coupled with the important characteristic of usability -- a necessary coupling if VEs are to reach their full potential. Although numerous methods exist for usability evaluation of interactive computer applications, these methods have well-known limitations, especially for evaluating VEs. Thus, there is a great need to develop usability evaluation methods and criteria specifically for VEs. Our goal is to increase awareness of the need for usability engineering of VEs and to lay a scientific foundation for developing high-impact methods for usability engineering of VEs. The first step in our multi-year research plan has been accomplished, yielding a comprehensive multi-dimensional taxonomy of usability characteristics specifically for VEs. This taxonomy was developed by collecting and synthesizing information from literature, conferences, World Wide Web (WWW) searches, investigative research visits to top VE facilities, and interviews of VE researchers and developers. The taxonomy consists of four main areas of usability issues: Users and User Tasks in VEs, general user and task characteristics and types of tasks in VEs

The rapidly growing interest in interactive threedimensional (3D) computer environments highly recommend the hand gesture as one of their interaction modalities. Among several factors constituting a hand gesture, hand movement pattern is spatiotemporally variable and informative, but its automatic recognition is not trivial. In this paper, we describe a hidden Markov(HMM)-based method for recognizing the space-time hand movement pattern. HMM models the spatial variance and the time-scale variance in the hand movement. As for the recognition of the continuous, connected hand movement patterns, HMMbased segmentation method is introduced. To deal with the dimensional complexity caused by the 3D problem space, the plane fitting method is employed and the 3D data is reduced into 2D. These 2D data are then encoded as the input to HMMs. In addition to the hand movement, which is regarded as the primary attribute of the hand gesture, we also consider the hand configuration(posture) and the p...

this paper, we develop and articulate a set of design principles for constructing - in a systematic and extensible manner - multi-hand gestures on touch surfaces that can sense multiple points and shapes, and can also accommodate conventional point-based input

ion Signal Level Semantic Level Figure 13: Multimodal Design Space (adapted from [224]) system in the design space is the pivotal center of its features. According to the characterization of an interaction along the two dimensions, fusion, and use of modalities, four basic types of multimodal interactions can be distinguished: alternative, synergistic, exclusive, and concurrent multimodal interaction, as shown in Figure 13. Obviously, synergistic systems subsume the other three classes of multimodal systems. Therefore, architectural models of multimodal integration (as presented in the next subsection and in Section 9) are sufficient if they are able to model synergistic cooperation of modalities. 6.2.2 Fusion of Multimodal Input Fusion of multimodal input events can occur on different levels, ranging from signal-level to semantic-level. Signal-level fusion (or lexical fusion [224]) performs the combination of multimodal input at the level of the input signal. Signal-level fusion has...

It is difficult to navigate virtual environment as in real world and to interact with other participant in virtual environment, especially wearing Head-Mounted Display (HMD). We developed Virtual Office Environment System (VOES), and avatar is used to navigate and to interact with other participants. For easy and intuitive control of avatar motion in the system, we use continuous hand gesture recognition system. State automata are proposed in hand gesture recognition to segment continuous hand gesture and to remove meaningless motion. Using avatar and gesture interface, this system provides natural navigation and interaction in virtual environment system.

Abstract. We present a scenario whereby an agent senses, interprets and copies a range of facial and gesture expression from a person in the real-world. Input is obtained via a video camera and processed initially using computer vision techniques. It is then processed further in a framework for agent perception, planning and behaviour generation in order to perceive, interpret and copy a number of gestures and facial expressions corresponding to those made by the human. By perceive, wemeanthat the copied behaviour may not be an exact duplicate of the behaviour made by the human and sensed by the agent, but may rather be based on some level of interpretation of the behaviour. Thus, the copied behaviour may be altered and need not share all of the characteristics of the original made by the human. 1

Abstract not found

Recent approaches to providing users with a more natural method of interacting with computer applications have shown that more than one mode of input can be both beneficial and intuitive as a communication medium between humans and computers. Two modalities in particular, whole-hand and speech input, represent a natural form of communication that has been ingrained in our physical and mental makeup since birth. In this thesis, we investigate the use of whole-hand and speech input in virtual environments in the context of two applications domains: scientific visualization and interior design. By examining the two modalities individually and in combination, and through the creation of two application prototypes (Multimodal Scientific Visualization Tool and Room Designer), we present anumber of contributions including a set of interface guidelines and interaction techniques for whole-hand and speech input.

In this paper we describe the experience of researchers and artists involved in the activities of the "computer ART lab " (cART lab) of the italian National Council of Research (C.N.R.) in Pisa, regarding the "wireless technology " developed for controlling in real-time and giving expression to interactive multimedia performances. Due to the daily increase in computers power and electronics systems able to sense the presence, the shape, the distance and the position of objects, a new field of investigation and implementation has been started in the last few years: computer recognition of human gesture [1][2]. As a result, the human body itself can now be considered as a natural and powerful expressive "interface " to give feeling to performances based on computer generated electro-acoustic music and computer generated visual-art. Modem human computer interfaces are extremely rich, incorporating traditional interface devices such as keyboard and mouse and a wealth of advanced media types: sound, video, animated graphics. The term multi-modal is often associated with such interfaces to emphasize that the combined use of multiple modes of perception is relevant to the user's interface [3][4]. The most relevant devices and systems developed at cART lab for gesture recognition to be used in interactive multimedia performances are here reported.

Gesture interaction is a versatile, intuitive way of interacting with computers, especially suited for virtual environments and applications where many degrees of freedom are important. Human gestures have many meanings and uses, and are often different from person to person, while gesture input devices often introduce artifacts; this makes recognizing and interpreting gestures nontrivial. Gestures can be divided into postures, where the configuration of the hand is static, and true gestures, where the flexion of the fingers and the hand position/direction are changing dynamically. A recognition model based on a hybrid articial neural network combining a radial basis function and a Bayesian classifier network is developed and tested with various forms of pre-processing or input representation. The results suggest that dynamic gesture recognition is feasible even for complex gestures, but that context information may be needed for reliable recognition.