This article introduces a new interaction model called Instrumental Interaction that extends and generalizes the principles of direct manipulation. It covers existing interaction styles, including traditional WIMP interfaces, as well as new interaction styles such as two-handed input and augmented reality. It defines a design space for new interaction techniques and a set of properties for comparing them. Instrumental Interaction describes graphical user interfaces in terms of domain objects and interaction instruments. Interaction between users and domain objects is mediated by interaction instruments, similar to the tools and instruments we use in the real world to interact with physical objects. The article presents the model, applies it to describe and compare a number of interaction techniques, and shows how it was used to create a new interface for searching and replacing text. Keywords Interaction model, WIMP interfaces, direct manipulation, post-WIMP interfaces, instrumental ...

We discuss a two-handed user interface designed to support three-dimensional neurosurgical visualization. By itself, this system is a “point design, ” an example of an advanced user interface technique. In this work, we argue that in order to understand why interaction techniques do or do not work, and to suggest possibilities for new techniques, it is important to move beyond point design and to introduce careful scientific measurement of human behavioral principles. In particular, we argue that the common-sense viewpoint that “two hands save time by working in parallel ” may not always be an effective way to think about two-handed interface design because the hands do not necessarily work in parallel (there is a structure to two-handed manipulation) and because two hands do more than just save time over one hand (two hands provide the user with more information and can structure how the user thinks about a task). To support these claims, we present an interface design developed in collaboration with neurosurgeons which has undergone extensive informal usability testing, as well as a pair of formal experimental studies which investigate behavioral aspects of two-handed virtual object manipulation. Our hope is that this discussion will help others to apply the lessons learned in our neurosurgery application to future two-handed user interface

Despite intense and wide-spread research in both virtual environments (VEs) and usability, the exciting new technology of VEs has not yet been closely coupled with the important characteristic of usability -- a necessary coupling if VEs are to reach their full potential. Although numerous methods exist for usability evaluation of interactive computer applications, these methods have well-known limitations, especially for evaluating VEs. Thus, there is a great need to develop usability evaluation methods and criteria specifically for VEs. Our goal is to increase awareness of the need for usability engineering of VEs and to lay a scientific foundation for developing high-impact methods for usability engineering of VEs. The first step in our multi-year research plan has been accomplished, yielding a comprehensive multi-dimensional taxonomy of usability characteristics specifically for VEs. This taxonomy was developed by collecting and synthesizing information from literature, conferences, World Wide Web (WWW) searches, investigative research visits to top VE facilities, and interviews of VE researchers and developers. The taxonomy consists of four main areas of usability issues: Users and User Tasks in VEs, general user and task characteristics and types of tasks in VEs

The literature has long suggested that the design of computer input devices should make use of the fine, smaller muscle groups and joints in the fingers, since they are richly represented in the human motor and sensory cortex and they have higher information processing bandwidth than other body parts. This hypothesis, however, has not been conclusively verified with empirical research. The present work studied such a hypothesis in the context of designing 6 degree-of-freedom (DOF) input devices. The work attempts to address both a practical need- designing efficient 6 DOF input devices- and the theoretical issue of muscle group differences in input control. Two alternative 6 DOF input devices, one including and the other excluding the fingers from the 6 DOF manipulation, were designed and tested in a 3D object docking experiment. Users ' task completion times were significantly shorter with the device that utilised the fingers. The results of this study strongly suggest that the shape and size of future input device designs should constitute affordances that invite finger participation in input control.

Previous research has shown that rotation and orientation of items plays three major roles during collaboration: comprehension, coordination and communication. Based on these roles of orientation and advice from kinesiology research, we have designed the Rotate’N Translate (RNT) interaction mechanism, which provides integrated control of rotation and translation using only a single touch-point for input. We present an empirical evaluation comparing RNT to a common rotation mechanism that separates control of rotation and translation. Results of this study indicate RNT is more efficient than the separate mechanism and better supports the comprehension, coordination and communication roles of orientation.

Eye gaze interaction can provide a convenient and natural addition to user-computer dialogues. We have previously reported on our interaction techniques using eye gaze [10]. While our techniques seemed useful in demonstration, we now investigate their strengths and weaknesses in a controlled setting. In this paper, we present two experiments that compare an interaction technique we developed for object selection based on a where a person is looking with the most commonly used selection method using a mouse. We find that our eye gaze interaction technique is faster than selection with a mouse. The results show that our algorithm, which makes use of knowledge about how the eyes behave, preserves the natural quickness of the eye. Eye gaze interaction is a reasonable addition to computer interaction and is convenient in situations where it is important to use the hands for other tasks. It is particularly beneficial for the larger screen workspaces and virtual environments of the future, and it will become increasingly practical as eye tracker technology matures.

We present experimental work that explores the factors governing symmetric bimanual interaction in a two-handed task that requires the user to track a pair of targets, one target with each hand. A symmetric bimanual task is a twohanded task in which each hand is assigned an identical role. In this context, we explore three main experimental factors. We vary the distance between the pair of targets to track: as the targets become further apart, visual diversion increases, forcing the user to divide attention between the two targets. We also vary the demands of the task by using both a slow and a fast tracking speed. Finally, we explore visual integration of sub-tasks: in one condition, the two targets to track are connected by a line segment which visually links the targets, while in the other condition there is no connecting line. Our results indicate that all three experimental factors affect the degree of parallelism, which we quantify using a new metric of bimanual parallelism. Howe...

This study investigates user performance when using semi-transparent tools in interactive 3D computer graphics environments. We hypothesize that when the user moves a semi-transparent surface in a 3D graphic display, the partial occlusion effect introduced through semi-transparency acts as an effective cue in target localization --- an essential component in many 3D manipulation tasks. This hypothesis was tested in a controlled experiment in which subjects were asked to acquire dynamic 3D targets (virtual fish) with a 3D cursor. In the experiment, cursors with and without semitransparent surfaces were compared in monoscopic and stereoscopic displays. Statistically significant effects for trial completion time, error rate and error magnitude were observed for stereopsis and partial occlusion. The partial occlusion cue was effectively used by subjects in both monoscopic and stereoscopic displays. It was no less effective than stereopsis for successful 3D target acquisition. Subjects' performance in each of the conditions improved with learning, but their relative ranking remained the same. Subjective evaluations also supported the conclusions drawn from performance measures. The experimental results and their implications are discussed, with emphasis on the relative, discrete nature of the semi-transparency cue and on interactions between depth cues. The paper concludes with a review of a number of existing and potential future applications of semi-transparency in human computer interaction.

High precision parameter manipulation tasks typically require adjustment of the scale of manipulation in addition to the parameter itself. This paper introduces the notion of Zoom Sliding, or Zliding, for fluid integrated manipulation of scale (zooming) via pressure input while parameter manipulation within that scale is achieved via x-y cursor movement (sliding). We also present the Zlider (Figure 1), a widget that instantiates the Zliding concept. We experimentally evaluate three different input techniques for use with the Zlider in conjunction with a stylus for x-y cursor positioning, in a high accuracy zoom and select task. Our results marginally favor the stylus with integrated isometric pressure sensing tip over bimanual techniques which separate zooming and sliding controls over the two hands. We discuss the implications of our results and present further designs that make use of Zliding.

Computing environments which involve many interacting devices are a challenge for system and user interface designers. A prototype of a multiple-device application consisting of a personal digital assistant (PDA) that operates in conjunction with interactive television (ITV) was developed from user requirements for a real estate information service. The application is used both as a stand-alone service and in conjunction with a television. Users interact exclusively with the PDA. The television responds to PDA output and is used for the presentation of visual images and videos. In this paper the application is described and user interface design issues that arise in the context of multiple device systems are discussed. Keywords: Personal digital assistants (PDA), interactive television (ITV), ubiquitous computing, mobile computing, multiple devices. DESIGNING FOR MULTIPLE DEVICES Future computing environments are likely to involve a combination of specialized personal devices that ...