This research explores distributed sensing techniques for mobile devices using synchronous gestures. These are patterns of activity, contributed by multiple users (or one user with multiple devices), which take on a new meaning when they occur together in time, or in a specific sequence in time. To explore this new area of inquiry, this work uses tablet computers augmented with touch sensors and twoaxis linear accelerometers (tilt sensors). The devices are connected via an 802.11 wireless network and synchronize their time-stamped sensor data. This paper describes a few practical examples of interaction techniques using synchronous gestures such as dynamically tiling together displays by physically bumping them together, discusses implementation issues, and speculates on further possibilities for synchronous gestures.

Recent advances in mobile computing allow the users to deal with 3D interactive graphics on handheld computers. Although the computing resources and screen resolutions grow steadily, user interfaces for handheld computers do not change significantly. Consequently, we designed a new 3-DOF interface adapted to the characteristics of handheld computers. This interface tracks the movement of a target that the user holds behind the screen by analyzing the video stream of the handheld computer camera. The position of the target is directly inferred from the color-codes that are printed on it using an efficient algorithm. The users can easily interact in realtime in a mobile setting. The visualization of the data is good as the target does not occlude the screen and the interaction techniques are not dependent on the orientation of the handheld computer. We used the interface in several test applications for the visualization of large images such as maps, the manipulation of 3D models, and the navigation in 3D scenes. This new interface favors the development of 2D and 3D interactive applications on handheld computers.

Abstract. The rich interaction capabilities of public terminals can make them more convenient to use than small personal devices, such as smart phones. However, the use of public terminals to handle personal data may compromise privacy. We present a system that enables users to access their applications and data securely using a combination of public terminals and a more trusted, personal device. Our system (i) provides users with capabilities to censor the public terminal display, so that it does not show private data; (ii) filters input events coming from the public terminal, so that maliciously injected keyboard/pointer events do not compromise privacy; and (iii) enables users to view personal information and perform data-entry via their personal device. A key feature of our system is that it works with unmodified applications. A prototype implementation of the system has been publicly released for Linux and Windows. The results arising from a pilot usability study based on this implementation are presented. 1

Abstract. Some ubiquitous computing visions propose to embed an abundance of input and output devices in the environment. Depending on the context, the user can either interact with her limited personal device, or input and output devices from the surrounding infrastructure. But what about situations in which the user has need for the capabilities of both types of devices at the same time? A user might need to display something privately, while needing the screen real estate of a large wall mounted display. For such situations we propose using mobile devices together with those provided by the environment. We have developed a runtime environment that coordinates multiple devices to do just that. Currently we are exploring the usability of the use of multiple devices concurrently. We are also investigating the problems of authoring such interfaces. 1.

How should a distributed file system manage access to protected content? On one hand, distributed storage should make data access pervasive: authorized users should be able to access their data from any location. On the other hand, content protection is designed to restrict access — this is often accomplished by limiting the set of computers from which content can be accessed. In this paper, we propose a new method for storing content in distributed storage called Cobalt. Rather than grant access to data based on the computer that reads the data, Cobalt grants access based on the physical proximity of authorized users. Protected content is stored encrypted in the distributed Blue File System; files can only be decrypted through the cooperation of a personal, mobile device such as cell phone. The Cobalt device is verified by content providers: it acts as a proxy that protects their interests by only decrypting data when policies specified during content acquisition are satisfied. Wireless communication with the device is used to determine the physical proximity of its user; when the Cobalt device moves out of range, protected content is made inaccessible. Our results show that Cobalt adds only modest overhead to content acquisition and playback, yet it enables new forms of interaction such as the ability to access protected content on ad hoc media players and create playlists that adapt to the tastes of nearby users. 1

Large information displays are common in public and semipublic spaces but still require rapid and lightweight ways for users to interact with them. We present BlueTone, a framework for developing large display applications which will interpret and react to dual-tone multi-frequency sounds transmitted from mobile phones paired with the display using the Bluetooth headset profile. BlueTone enables text entry, cursor manipulation and menu selection without requiring the installation of any special software on a user’s mobile phone. Author Keywords BlueTone, public display, at a distance interaction, Bluetooth

When the user is engaged with a real-world task it can be inappropriate or difficult to use a smartphone. To address this concern, we developed ShoeSense, a wearable system consisting in part of a shoe-mounted depth sensor pointing upward at the wearer. ShoeSense recognizes relaxed and discreet as well as large and demonstrative hand gestures. In particular, we designed three gesture sets (Triangle, Radial, and Finger-Count) for this setup, which can be performed without visual attention. The advantages of ShoeSense are illustrated in five scenarios: (1) quickly performing frequent operations without reaching for the phone, (2) discreetly performing operations without disturbing others, (3) enhancing operations on mobile devices, (4) supporting accessibility, and (5) artistic performances. We present a proof-of-concept, wearable implementation based on a depth camera and report on a lab study comparing social acceptability, physical and mental demand, and user preference. A second study demonstrates a 94-99% recognition rate of our recognizers.

Within the last decade, mobile devices have become an integral part of society, at home or work, in industrialized and developing countries. For children, these devices have primarily been geared towards communication, information consumption, or individual creative purposes. Prior research indicates social interaction and collaboration are essential to the social and cognitive development of young children. This dissertation research focuses on supporting collaboration among mobile users, specifically children ages 6 to 10 — while collaboratively reading and creating stories. I developed Mobile Stories, a novel software system for the Windows Mobile platform that supports collaborative story experiences, with special attention to two collocated collaboration experiences: content splitting and space sharing. Content splitting is where interface parts (e.g. words, pictures) are split between two or more devices. Space sharing is where the same content (e.g. a document) is spread or shared across devices. These collocated collaborative configurations help address mobile devices ’ primary limitation: a small screen. The three research questions addressed are: how does Mobile Stories affect