Current input device taxonomies and other frameworks typically emphasize the mechanical structure of input devices. We suggest that selecting an appropriate input device for an interactive task requires looking beyond the physical structure of devices to the deeper perceptual structure of the task, the device, and the interrelationship between the perceptual structure of the task and the control properties of the device. We atllrm that perception is key to understanding performance of multidimensional input devices on multidimensional tasks. We have therefore extended the theory of processing of perceptual structure to graphical interactive tasks and to the control structure of input devices. This allows us to predict task and device combinations that lead to better performance and hypothesize that performance is improved when the perceptual structure of the task matches the control structure of the device. We conducted an experiment in which subjects performed two tasks with different perceptual structures, using two input devices with correspondingly different control structures, a three-dimensional tracker and a mouse. We analyzed both speed and accuracy, as well as the trajectories generated by subjects as they used the unconstrained three-dimensional tracker to perform each task. The results support our hypothesis and confirm the importance of matching the perceptual structure of the task and the control structure of the input device. Categories and Subject Descriptors: H.1.2 [Models and Principles]: User/Machine

The iStuff toolkit of physical devices, and the flexible software infrastructure to support it, were designed to simplify the exploration of novel interaction techniques in the post-desktop era of multiple users, devices, systems and applications collaborating in an interactive environment. The toolkit leverages an existing interactive workspace infrastructure, making it lightweight and platform independent. The supporting software framework includes a dynamically configurable intermediary to simplify the mapping of devices to applications. We describe the iStuff architecture and provide several examples of iStuff, organized into a design space of ubiquitous computing interaction components. The main contribution is a physical toolkit for distributed, heterogeneous environments with run-time retargetable device data flow. We conclude with some insights and experiences derived from using this toolkit and framework to prototype experimental interaction techniques for ubiquitous computing environments.

In these days of multimodal systems and interfaces, many research teams are investigating the purposes for which novel combinations of modalities can be used. It is easy to forget that we still lack solid foundations for evaluating the functionality of individual families of input/output modalities, such as the speech modalities. The reason why these foundations are missing is the complexity of the problem. Based on the study of particular applications, empirical investigations of speech functionality address points in a vast multi-dimensional design space. At best, solid findings yield low-level generalisations which can be used by designers developing almost identical applications...

Pre-screen projection is a new interaction technique that allows a user to pan and zoom integrally through a scene simply by moving his or her head relative to the screen. The underlying concept is based on real-world visual perception, namely, the fact that a person's view changes as the head moves. Pre-screen projection tracks a user's head in three dimensions and alters the display on the screen relative to head position, giving a natural perspective effect in response to a user's head movements. Specifically, projection of a virtual scene is calculated as if the scene were in front of the screen. As a result, the visible scene displayed on the physical screen expands (zooms) dramatically as a user moves nearer. This is analogous to the real world, where the nearer an object is, the more rapidly it visually expands as a person moves toward it. Further, with pre-screen projection a user can navigate (pan and zoom) around a scene integrally, as one unified activity, rather than performing panning and zooming as separate tasks. This paper describes the technique, the real-world metaphor on which it is conceptually based, issues involved in iterative development of the technique, and our approach to its empirical evaluation in a realistic application testbed.

This article introduces a design framework for sensing-based interfaces in which designers are encouraged to compare expected physical movements with those that can be sensed by a computer system and those that are desired by a particular application. They are asked to treat the boundaries between these as interesting areas of the design space, both in terms of problems to be solved and also opportunities to be exploited. Our framework is motivated by four recent trends in human computer interaction (HCI)

The study of human-computer interaction within immersive virtual environments requires us to balance what we have learned from the design and use of desktop interfaces with novel approaches that allow us to work effectively in three dimensions. This dissertation presents empirical results from four studies into different techniques for indirect manipulation in immersive virtual environments. These studies use a testbed called the Haptic Augmented Reality Paddle (or HARP) system to compare different immersive interaction techniques. The results show that the use of hand-held windows as an interaction technique can improve performance and preference on tasks requiring head movement. Also, the use of a physical prop registered with the visual representation of an interaction surface can significantly improve user performance and preference compared to having no physical surface. Furthermore, even if a physical surface is not present, constraining user movement for manipulating interface widgets can also improve performance. Research into defining and classifying interaction techniques in the form of a taxonomy for interaction in immersive virtual environments is also presented. The taxonomy classifies

Information visualization is an experience in which both the aesthetic representations and interaction are part. Such an experience can be augmented through close consideration of its major components. Interaction is crucial to the experience, yet it has seldom been adequately explored in the field. We claim that direct mediated interaction can augment such an experience. This paper discusses the reasons behind such a claim and proposes a mediated interactive manipulation scheme based on the notion of directness. It also describes the ways in which such a claim will be validated. The Literature Knowledge Domain (LKD) is used as the concrete domain around which the discussions will be held.

This report proposes a methodology for selecting appropriate interaction techniques and devices which meet the motor, sensory, communication and cognitive abilities of different user groups. The proposed solutions are targeted to B-ISDN services and assume the multimedia capabilities of a broadband network environment. The methodology mainly concerns the User Interface (UI) part which handles the low level (i.e lexical) interaction issues - i.e. those related to the external representation of service information and to the UI properties which are directly viewed/manipulated by the user. It incorporates a theoretical investigation of lexical interaction issues together with the results of a practical evaluation of some of the identified lexical interaction possibilities. The theoretical work identifies some important interaction tasks and their associated techniques/devices, as a basic set of design elements on which lexical interaction in a multimedia/multimodal environment can be buil...

T9DTryrp# consists of a small knowledge-based system teaching and assisting designers of interactive applications in selecting appropriate interaction styles for a particular context of use. As any other tool for working with guidelines, five development milestones have been browsed. Guidelines for selecting interaction styles have been captured in a knowledge-base system as selections among an available set of possible interaction styles, a set of parameters characterizing the context of use, questions to provide the parameters' values, and selection rules. Each rule selects a candidate interaction style candidate according to values assigned to parameters of the context of use according to a rule-based language, which basically consists of a first-order predicate logic formula. The values of these parameters are either stored or prompted to the designer through questions to form a final set of possible interaction styles.

Designing wearable application interfaces that integrate well into real-world processes like aircraft maintenance or medical examinations is challenging. One of the main success criteria is how well the multimodal interaction with the computer system fits an already existing real-world task. Therefore, the interface design needs to take the real-world task flow into account from the beginning. We propose a model of interaction devices and human interaction capabilities that helps evaluate how well different interaction devices/techniques integrate with specific realworld scenarios. The model was developed based on a survey of wearable interaction research literature. Combining this model with descriptions of observed real-world tasks, possible conflicts between task performance and device requirements can be visualized helping the interface designer to find a suitable solution.