This dissertation defines and explores Graspable User Interfaces, an evolution of the input mechanisms used in graphical user interfaces (GUIs). A Graspable UI design provides users concurrent access to multiple, specialized input devices which can serve as dedicated physical interface widgets, affording physical manipulation and spatial arrangements. Like conventional GUIs, physical devices function as “handles” or manual controllers for logical functions on widgets in the interface. However, the notion of the Graspable UI builds on current practice in a number of ways. With conventional GUIs, there is typically only one graphical input device, such as a mouse. Hence, the physical handle is necessarily “time-multiplexed,” being repeatedly attached and unattached to the various logical functions of the GUI. A significant aspect of the Graspable UI is that there can be more than one input device. Hence input control can then be “space-multiplexed.” That is, different devices can be attached to different functions, each independently (but possibly simultaneously) accessible. This, then affords the capability to take advantage of the

This article describes a theory of the computations underlying the selection of coordinated motion patterns, especially in reaching tasks. The central idea is that when a spatial target is selected as an object to be reached, stored postures are evaluated for the contributions they can make to the task. Weights are assigned to the stored postures, and a single target posture is found by taking a weighted sum of the stored postures. Movement is achieved by reducing the distance between the starting angle and target angle of each joint. The model explains compensation for reduced joint mobility, tool use, practice effects, performance errors, and aspects of movement kinematics. Extensions of the model can account for anticipation and coarticulation effects, movement through via points, and hierarchical control of series of movements. The goal of this research is a unified theory of the planning and control of physical action. Such a theory, as several authors have noted (Jeannerod, in press; Rosenbaum, 1991; Wing, 1993), has been lacking. Instead, specialized models have been designed to account for data from different tasks. The sentiment

Conventional methods for programming a robot either are inflexible or demand significant expertise. While the notion of automatic programming by high-level goal specification addresses these issues, the overwhelming complexity of planning manipulator grasps and paths remains a formidable obstacle to practical implementation. This thesis describes the approach of programming a robot by human demonstration. Our system observes a human performing the task, recognizes the human grasp, and maps it onto the manipulator. Using human actions to guide robot execution greatly reduces the planning complexity. In analyzing the task sequence, the system first divides the observed sensory data into meaningful temporal segments, namely the pregrasp, grasp, and manipulation phases. This is achieved by analyzing the human hand motion profiles. The features used are the fingertip polygon area (the fingertip polygon being the polygon whose vertices are the fingertips), hand speed, and the volume sweep r...

Inertial measurement components, which sense either acceleration or angular rate, are being embedded into common user interface devices more frequently as their cost continues to drop dramatically. These devices hold a number of advantages over other sensing technologies: they measure relevant parameters for human interfaces and can easily be embedded into wireless, mobile platforms. The work in this dissertation demonstrates that inertial measurement can be used to acquire rich data about human gestures, that we can derive efficient algorithms for using this data in gesture recognition, and that the concept of a parameterized atomic gesture recognition has merit. Further we show that a framework combining these three levels of description can be easily used by designers to create robust applications.

Abstract not found

This work explores how the availability of visual and haptic feedback affects the kinematics of reaching performance in a tabletop virtual environment. Eight subjects performed reach-to-grasp movements toward target objects of various sizes in conditions where visual and haptic feedback were either present or absent. It was found that movement time was slower when visual feedback of the moving limb was not available. Further MT varied systematically with target size when haptic feedback was available (i.e. augmented targets), and thus followed Fitts ’ law. However, movement times were constant regardless of target size when haptic feedback was removed. In depth analysis of the reaching kinematics revealed that subjects spent longer decelerating toward smaller targets in conditions where haptic feedback was available. In contrast, deceleration time was constant when haptic feedback was absent. These results suggest that visual feedback about the moving limb and veridical haptic feedback about object contact are extremely important for humans to effectively work in virtual environments. KEYWORDS Augmented reality, kinematic data, object manipulation, haptic feedback, visual feedback, sensory information, human performance, Fitts ’ law, empirical data, interaction

Abstract. Common movements like reaching, striking, etc. observed during surveillance have highly variable target locations. This puts appearance-based techniques at a disadvantage for modelling and recognizing them. Psychological studies indicate that these actions are ballistic in nature. Their trajectories have simple structures and are determined to a great degree by the starting and ending positions. We present an approach for movement recognition that explicitly considers their ballistic nature. This enables the decoupling of recognition from the movement’s trajectory, allowing generalization over a range of target-positions. A given movement is first analyzed to determine if it is ballistic. Ballistic movements are further classified into reaching, striking, etc. The proposed approach was tested with motion capture data obtained from the CMU MoCap database. 1

Abstract—The principle of contralateral organization of the visual and motor systems was exploited to investigate contributions of the cerebral hemispheres to the mental representation of prehension in healthy, right-handed human subjects. Graphically rendered dowels were presented to either the left or right visual field in a variety of different orientations, and times to determine whether an underhand or overhand grip would be preferred for engaging these stimuli were measured. Although no actual reaching movements were performed, a significant advantage in grip-selection time was found when information was presented to the cerebral hemisphere contralateral to the designated response hand. Results are consistent with the position that motor imagery recruits neurocognitive mechanisms involved in movement planning. More precisely, these findings indicate that processes within each cerebral hemisphere participate in mentally representing object-oriented actions of the contralateral hand. An important contribution to resolving the long-standing debate over the relationship between imagery and perception has been made by numerous studies indicating that the two categories of behavior involve common neural substrates (for comprehensive reviews, see

This article includes two experiments aimed at investigating how two-dimensional (2-D) and threedimensional (3-D) distractors affect the kinematics of prehension and aiming movements in order to understand the attentional processes involved in visuomotor control. In Experiment 1, subjects grasped large targets in the presence of both large and small 3-D distractors and their corresponding 2-D photographs. The distractors appeared for either 10 sec or appeared simultaneously with the target presentation. It was found that reach and grasp kinematics were influenced primarily by the small, suddenly appearing 3-D distractors. In Experiment 2, the purpose was to examine the conclusion that competition between objects (target and distractor) is related to the behavioral goal of the task. Experiment 2 is a replication of Experiment 1, with the exception that pointing movements were made instead of grasping movements. Results show that both 3-D and 2-D distractors interfered with pointing kinematics, supporting the hypothesis that attentional mechanisms are related to the goal of the task. This research was sponsored by the Australian Research Council. I thank T. Sanocki, H. A. Sedgwick, K. Shapiro, and an anonymous referee for valuable comments and suggestions for improving the quality of the article. Correspondence concerning this article should be addressed

Abstract. Prehension, or reaching-to-grasp is a very common movement performed by users of Tangible User Interfaces (TUIs) because through this movement users can manipulate tangible artifacts. Here we present an experiment that provides evidence towards the hypothesis that prehensile movements can be modeled based on the amplitude of the prehension movement. We then explore consequences of this evidence on the modeling and evaluation of TUIs using tools that can predict task completion times, such as