MouseLight is a spatially-aware standalone mobile projector with the form factor of a mouse that can be used in combination with digital pens on paper. By interacting with the projector and the pen bimanually, users can visualize and modify the virtually augmented contents on top of the paper, and seamlessly transition between virtual and physical information. We present a high fidelity hardware prototype of the system and demonstrate a set of novel interactions specifically tailored to the unique properties of MouseLight. MouseLight differentiates itself from related systems such as PenLight in two aspects. First, MouseLight presents a rich set of bimanual interactions inspired by the ToolGlass interaction metaphor, but applied to physical paper. Secondly, our system explores novel displaced interactions, that take advantage of the independent input and output that is spatially aware of the underneath paper. These properties enable users to issue remote commands such as copy and paste or search. We also report on a preliminary evaluation of the system, which produced encouraging observations and feedback.

Figure 1: The 1Line Keyboard. It consists of only eight character keys, flick gestures, and a novel approach for integrating the spacebar into the bezel. It is 140px tall (26.9mm) and 39.8 % of the size of the native landscape iPad keyboard. Current soft QWERTY keyboards often consume a large portion of the screen space on portable touchscreens. This space consumption can diminish the overall user experience on these devices. In this paper, we present the 1Line keyboard, a soft QWERTY keyboard that is 140 pixels tall (in landscape mode) and 40 % of the height of the native iPad QWERTY keyboard. Our keyboard condenses the three rows of keys in the normal QWERTY layout into a single line with eight keys. The sizing of the eight keys is based on users ’ mental layout of a QWERTY keyboard on an iPad. The system disambiguates the word the user types based on the sequence of keys pressed. The user can use flick gestures to perform backspace and enter, and tap on the bezel below the keyboard to input a space. Through an evaluation, we show that participants are able to quickly learn how to use the 1Line keyboard and type at a rate of over 30 WPM after just five 20-minute typing sessions. Using a keystroke level model, we predict the peak expert text entry rate with the 1Line keyboard to be 66-68 WPM. ACM Classification: H5.2 [Information interfaces and presentation]: User Interfaces – Input devices and strategies

Recognizing possibly thousands of objects is a crucial capability for an autonomous agent to understand and interact with everyday environments. Practical object recognition comes in multiple forms: Is this a coffee mug? (category recognition). Is this Alice’s coffee mug? (instance recognition). Is the mug with the handle facing left or right? (pose recognition). We present a scalable framework, Object-Pose Tree, which efficiently organizes data into a semantically structured tree. The tree structure enables both scalable training and testing, allowing us to solve recognition over thousands of object poses in near real-time. Moreover, by simultaneously optimizing all three tasks, our approach outperforms standard nearest neighbor and 1-vs-all classifications, with large improvements on pose recognition. We evaluate the proposed technique on a dataset of 300 household objects collected using a Kinect-style 3D camera. Experiments demonstrate that our system

Pico projectors attached to mobile phones allow users to view phone content using a large display. However, to provide input to projector phones, users have to look at the device, diverting their attention from the projected image. Additionally, other collocated users have no way of interacting with the device. We present ShadowPuppets, a system that supports collocated interaction with mobile projector phones. Shadow-Puppets allows users to cast hand shadows as input to mobile projector phones. Most people understand how to cast hand shadows, which provide an easy input modality. Additionally, they implicitly support collocated usage, as nearby users can cast shadows as input and one user can see and understand another user’s hand shadows. We describe the results of three user studies. The first study examines what hand shadows users expect will cause various effects. The second study looks at how users perceive hand shadows, examining what effects they think various hand shadows will cause. Finally, we present qualitative results from a study with our functional prototype and discuss design implications for systems using shadows as input. Our findings suggest that shadow input can provide a natural and intuitive way of interacting with projected interfaces and can support collocated collaboration. Author Keywords Projector-camera system, mobile projector phone, shadow,

Despite the prominence of multi-touch technologies, there has been little work investigating its integration into the desktop environment. Bringing multi-touch into desktop computing would give users an additional input channel to leverage, enriching the current interaction paradigm dominated by a mouse and keyboard. We provide two main contributions in this domain. First, we describe the results from a study we performed, which systematically evaluates the various potential regions within the traditional desktop configuration that could become multi-touch enabled. The study sheds light on good or bad regions for multi-touch, and also the type of input most appropriate for each of these regions. Second, guided by the results from our study, we explore the design space of multi-touch-integrated desktop experiences. A set of new interaction techniques are coherently integrated into a desktop prototype, called Magic Desk, demonstrating potential uses for multi-touch enabled desktop configurations.

This document provides a literature survey of papers published in the area of Human Machine Interaction(HMI), mainly focusing on state of the art technology that has or will have a major influence on the future direction of HMI. Through a glimpse of what the current technical achievements are and what the future might hold, it becomes clear that advanced HMI technologies will begin to converge, HMI devices will combine functionality, new levels of sensor fusions will be created, and all this for one purpose, to bring intelligent machines and robots more close to humans. HMI plays a key role for robots and machines to be used in the general public who have no knowledge of software and hardware, in particular

MouseLight is a spatially-aware aware standalone mobile projector with the form factor of a mouse that can be used in combination with digital pens on paper. By interacting with the projector and the pen bimanually, users can visualize and modify the virtually augmented ugmented contents on top of the paper, and seamlessly transition between virtual and physical information. We present a high fidelity hardware prototype of the system and demonstrate a set of novel interactions specifically tailored to the unique properties of MouseLight. MouseLight differentiates itself from related systems such as PenLight in two aspects. First, MouseLight presents a rich set of bimanual interactions inspired by the ToolGlass interaction metaphor, but applied to physical paper. Secondly, our system explores novel displaced interactions, that take advantage of the independent input and output that is spatially aware of the underneath paper. These properties enable users to issue remote commands such as copy and paste or search. We also report rt on a preliminary evaluation of the system, which produced encouraging observations and feedback.

In everyday office work, people smoothly use the space on their physical desks to work with documents of interest, and to keep associated tools and materials nearby for easy use. In contrast, the limited screen space of computer displays imposes interface constraints. Associated material is either placed off-screen (i.e., temporarily hidden) and requires extra work to access (window switching, menu selection) or crowds and competes with the work area (e.g., as palettes and icons). This problem is worsened by the increasing popularity of small displays such as tablets and laptops. To mitigate this problem, we investigate how we can exploit an unadorned physical desk space as an additional input canvas. Our Unadorned Desk detects coarse hovering over and touching of areas on an otherwise standard physical desk, which is used as input to the desktop computer. Unlike other augmented desks, feedback is given on the computer’s screen instead of on the desk itself. To better understand how people make use of this new input space, we conducted two user studies: (1) placing and retrieving application icons onto the desk, and (2) retrieving items from a predefined grid. We found that participants organize items in a grid for easier access, and are generally faster without affecting accuracy without on-screen feedback for few items, but were more accurate (though slower as they relied on feedback) for many items.