Toolglass ™ widgets are new user interface tools that can appear, as though on a transparent sheet of glass, between an application and a traditional cursor. They can be positioned with one hand while the other positions the cursor. The widgets provide a rich and concise vocabulary for operating on application objects. These widgets may incorporate visual filters, called Magic Lens™ filters, that modify the presentation of application objects to reveal hidden information, to enhance data of interest, or to suppress distracting information. Together, these tools form a see-through interface that offers many advantages over traditional controls. They provide a new style of interaction that better exploits the user’s everyday skills. They can reduce steps, cursor motion, and errors. Many widgets can be provided in a user interface, by designers and by users, without requiring dedicated screen space. In addition, lenses provide rich context-dependent feedback and the ability to view details and context simultaneously. Our widgets and lenses can be combined to form operation and viewing macros, and can be used over multiple applications.

Current interactive user interface construction tools are often more of a hindrance than a benefit during the early stages of user interface design. These tools take too much time to use and force designers to specify more of the design details than they wish at this early stage. Most interface designers, especially those who have a background in graphic design, prefer to sketch early interface ideas on paper or on a whiteboard. We are developing an interactive tool called SILK that allows designers to quickly sketch an interface using an electronic pad and stylus. SILK preserves the important properties of pencil and paper: a rough drawing can be produced very quickly and the medium is very flexible. However, unlike a paper sketch, this electronic sketch is interactive and can easily be modified. In addition, our system allows designers to examine, annotate, and edit a complete history of the design. When the designer is satisfied with this early prototype, SILK can transform the sket...

This paper presents an application that uses hand gesture input to control a computer while giving a presentation. In order to develop a prototype of this application, we have defined an interaction model, a notation for gestures, and a set of guidelines to design gestural command sets. This works aims to define interaction styles that work in computerized reality environments. In our application, gestures are used for interacting with the computer as well as for communicating with other people or operating other devices. KEYWORDS Hand gesture input, Interaction model, Computerized reality, Remote controlled user interfaces, Computer-aided presentations.

Current input device taxonomies and other frameworks typically emphasize the mechanical structure of input devices. We suggest that selecting an appropriate input device for an interactive task requires looking beyond the physical structure of devices to the deeper perceptual structure of the task, the device, and the interrelationship between the perceptual structure of the task and the control properties of the device. We atllrm that perception is key to understanding performance of multidimensional input devices on multidimensional tasks. We have therefore extended the theory of processing of perceptual structure to graphical interactive tasks and to the control structure of input devices. This allows us to predict task and device combinations that lead to better performance and hypothesize that performance is improved when the perceptual structure of the task matches the control structure of the device. We conducted an experiment in which subjects performed two tasks with different perceptual structures, using two input devices with correspondingly different control structures, a three-dimensional tracker and a mouse. We analyzed both speed and accuracy, as well as the trajectories generated by subjects as they used the unconstrained three-dimensional tracker to perform each task. The results support our hypothesis and confirm the importance of matching the perceptual structure of the task and the control structure of the input device. Categories and Subject Descriptors: H.1.2 [Models and Principles]: User/Machine

In this paper we challenge the assumption that an electronic instrument consists solely of an interface and a sound generator. We emphasise the importance of the mapping between input parameters and system parameters, and claim that this can define the very essence of an instrument.

Interest in pen-based user interfaces is growing rapidly. One potentially useful feature of pen-based user interfaces is gestures, that is, a mark or stroke that causes a command to execute. Unfortunately, it is difficult to design gestures that are easy 1) for computers to recognize and 2) for humans to learn and remember. To investigate these problems, we built a prototype tool for designing gesture sets. An experiment was then performed to gain insight into the gesture design process and to evaluate the tool. The experiment confirmed that gesture design is very difficult and suggested several ways in which current tools can be improved. The most important of these improvements is to make the tools more active and provide more guidance for designers. This paper describes the gesture design tool, the experiment, and its results. Keywords pen-based user interface, PDA, user study, gesture, UI design INTRODUCTION This work explores the process of gesture design with the goal of impr...

Our physical bodies play a central role in shaping human experience in the world, understanding of the world, and interactions in the world. This paper draws on theories of embodiment — from psychology, sociology, and philosophy — synthesizing five themes we believe are particularly salient for interaction design: thinking through doing, performance, visibility, risk, and thick practice. We introduce aspects of human embodied engagement in the world with the goal of inspiring new interaction design approaches and evaluations that better integrate the physical and computational worlds. Author Keywords Embodiment, bodies, embodied interaction, ubiquitous

this paper, we develop and articulate a set of design principles for constructing - in a systematic and extensible manner - multi-hand gestures on touch surfaces that can sense multiple points and shapes, and can also accommodate conventional point-based input

Movements of interfaces can be analysed in terms of whether they are sensible, sensable and desirable. Sensible movements are those that users naturally perform; sensable are those that can be measured by a computer; and desirable movements are those that are required by a given application. We show how a systematic comparison of sensible, sensable and desirable movements, especially with regard to how they do not precisely overlap, can reveal potential problems with an interface and also inspire new features. We describe how this approach has been applied to the design of three interfaces: the Augurscope II, a mobile augmented reality interface for outdoors; the Drift Table, an item of furniture that uses load sensing to control the display of aerial photographs; and pointing flashlights at walls and posters in order to play sounds.

This research investigates children’s use of two common mouse interaction styles, drag-anddrop and point-and-click, to determine whether the choice of interaction style impacts children’s performance in interactive learning environments. The interaction styles were experimentally compared to determine if either method was superior to the other in terms of speed, error rate, or user preference, for children. The two interaction styles were also compared based on children’s achievement and motivation, within a commercial software environment. Experiment I used an interactive learning environment as children played two versions of an educational puzzle-solving game, each version utilizing a different mouse interaction style; Experiment II used a mouse-controlled software environment modeled after the educational game. The results were similar to previous results reported for adults: the point-and-click interaction style was faster; fewer errors were committed using it; and it was preferred over the drag-and-drop interaction style. Within the context of the puzzle-solving game, the children solved significantly fewer puzzles, and they were less motivated using the version that utilized a drag-and-drop interaction style as compared to the version that utilized a point-and-click interaction style. These results were also explored through the use of state-transition diagrams and GOMS models, both of which supported the experimental data gathered.