This paper addresses the problem that a listener experiences when attempting to capture information presented during a lecture, meeting, or interview. Listeners must divide their attention between the talker and their notetaking activity. We propose a new device -- the Audio Notebook -- for taking notes and interacting with a speech recording. The Audio Notebook is a combination of a digital audio recorder and paper notebook, all in one device. Audio recordings are structured using two techniques: user structuring based on notetaking activity, and acoustic structuring based on a talker's changes in pitch, pausing, and energy. A field study showed that the interaction techniques enabled a range of usage styles, from detailed review to high speed skimming. The study motivated the addition of phrase detection and topic suggestions to improve access to the audio recordings. Through these audio interaction techniques, the Audio Notebook defines a new approach for navigation in the audio domain.

We motivate the use of desktop assistants in the context of web surfing and show how such a tool may be used to support activities in both cooperative and personal surfing. By cooperative surfing we mean surfing by a community of users who choose to cooperatively and asynchronously build up knowledge structures relevant to their group. Specifically, we describe the design of an assistant called Vistabar, which lives on the Windows desktop and operates on the currently active web browser. Vistabar instances working for individual users support the authoring of annotations and shared bookmark hierarchies, and work with profiles of community interests to make findings highly available. Thus, they support a form of community memory. Vistabar also serves as a form of personal memory by indexing pages the user sees to assist in recall. We present rationale for the assistant's design, describe roles it could play to support surfing (including those mentioned above), and suggest efficient impl...

This paper describes two interface prototypes which we have developed on our augmented desk interface system, EnhancedDesk. The first application is Interactive Textbook, which is aimed at providing an effective learning environment. When a student opens a page which describes experiments or simulations, Interactive Textbook automatically retrieves digital contents from its database and projects them onto the desk. Interactive Textbook also allows the student hands-on ability to interact with the digital contents. The second application is the Interactive Venn Diagram, which is aimed at supporting effective information retrieval. Instead of keywords, the system uses real objects such as books or CDs as keys for retrieval. The system projects a circle around each book; data corresponding the book are then retrieved and projected inside the circle. By moving two or more circles so that the circles intersect each other, the user can compose a Venn diagram interactively on the desk. We also describe the new technologies introduced in EnhancedDesk which enable us to implement these applications.

We present Bonfire, a self-contained mobile computing system that uses two laptop-mounted laser micro-projectors to project an interactive display space to either side of a laptop keyboard. Coupled with each micro-projector is a camera to enable hand gesture tracking, object recognition, and information transfer within the projected space. Thus, Bonfire is neither a pure laptop system nor a pure tabletop system, but an integration of the two into one new nomadic computing platform. This integration (1) enables observing the periphery and responding appropriately, e.g., to the casual placement of objects within its field of view, (2) enables integration between physical and digital objects via computer vision, (3) provides a horizontal surface in tandem with the usual vertical laptop display, allowing direct pointing and gestures, and (4) enlarges the input/output space to enrich existing applications. We describe Bonfire’s architecture, and offer scenarios that highlight Bonfire’s advantages. We also include lessons learned and insights for further development and use. ACM Classification Keywords H.5.2. [Information interfaces and presentation]: User

directories, etc---exist in both a physical (paper) and virtual (Web) form. Few approaches to knowledgment management and digital libraries fully exploit the opportunities a#orded by this fact. Motivated by the goal of seamless integration of physical artifacts and their Web counterparts, we describe a large-scale case study of one aspect of this relationship. Based on a corpus of hundreds of real-world product catalogs, we measure the e#ectiveness of hand-held scanner /OCR devices for the task of automatically retrieving a catalog's authoritative Web counterpart (the vendor's home page). We find that, despite OCR errors, text fragments scanned from product catalogs can serve as reasonably effective queries for retrieving the Web counterparts. Furthermore, the e#ectiveness of the technique increases with multiple scanned text fragments. Our main technical contribution is a novel machine learning approach to adaptively merging the retrieved documents from multiple scans.

The Escritoire is a horizontal desk interface that uses two projectors to create a foveal display. Items such as images, documents, and the interactive displays of other conventional computers, can be manipulated on the desk using pens in both hands. The periphery covers the desk, providing ample space for laying out the objects relevant to a task, allowing them to be identified at a glance and exploiting human spatial memory for rapid retrieval. The fovea is a high resolution focal area that can be used to view any item in detail. The projected images are continuously warped with commodity graphics hardware before display, to reverse the effects of misaligned projectors and ensure registration between fovea and periphery. The software is divided into a hardware-specific client driving the display, and a platform-independent server imposing control.

This thesis develops new approaches for people to physically represent and interact with aggregates of digital information. These support the concept of Tangible User Interfaces (TUIs), a genre of human-computer interaction that uses spatially reconfigurable physical objects as representations and controls for digital information. The thesis supports the manipulation of information aggregates through systems of physical tokens and constraints. In these interfaces, physical tokens act as containers and parameters for referencing digital information elements and aggregates. Physical constraints are then used to map structured compositions of tokens onto a variety of computational interpretations. This approach is supported through the design and implementation of several systems. The mediaBlocks system enables people to use physical blocks to “copy and paste ” digital media between specialized