Reading frequently involves not just looking at words on a page, but also underlining, highlighting and commenting, either on the text or in a separate notebook. This combination of reading with critical thinking and learning is called active reading [2]. To explore the premise that computation can enhance active reading we have built the XLibris "active reading machine." XLibris uses a commercial high-resolution pen tablet display along with a paper-like user interface to support the key affordances of paper for active reading: the reader can hold a scanned image of a page in his lap and mark on it with digital ink. To go beyond paper, XLibris monitors the free-form ink annotations made while reading, and uses these to organize and to search for information. Readers can review, sort and filter clippings of their annotated text in a "Reader's Notebook." XLibris also searches for material related to the annotated text, and displays links to similar documents unobtrusively in the margin. XLibris demonstrates that computers can help active readers organize and find information while retaining many of the advantages of reading on paper. Keywords Paper-like user interface, reading online, affordances of paper, pen computing, dynamic hypertext, document metaphor, information retrieval

"Linking by inking" is a new interface for reader-directed link construction that bridges reading and browsing activities. We are developing linking by inking in XLibris, a hypertext system based on the paper document metaphor. Readers use a pen computer to annotate page images with free-form ink, much as they would on paper, and the computer constructs hypertext links based on the ink marks. This paper proposes two kinds of readerdirected links: automatic and manual. Automatic links are created in response to readers' annotations. The system extracts the text near free-form ink marks, uses these terms to construct queries, executes queries against a collection of documents, and unobtrusively displays links to related documents in the margin or as "further reading lists." We also present a design for manual (ad hoc) linking: circling an ink symbol generates a multi-way link to other instances of the same symbol.

Increasingly, documents exist primarily in digital form. System designers have recently focused on making it easier to read digital documents, with annotation as an important new feature. But supporting annotation well is difficult because digital documents are frequently modified, making it challenging to correctly reposition annotations in modified versions. Few systems have addressed this issue, and even fewer have approached the problem from the users' point of view. This paper reports the results of two studies examining user expectations for "robust" annotation positioning in modified documents. We explore how users react to lost annotations, the relationship between types of document modifications and user expectations, and whether users pay attention to text surrounding their annotations. Our results could contribute substantially to effective digital document annotation systems. Keywords Annotation, robust, digital, documents, annotation system design. 1 INTRODUCTION Four ...

This paper presents a novel image editing program emphasizing easy selection and manipulation of material found in informal, casual documents such as sketches, handwritten notes, whiteboard images, screen snapshots, and scanned documents. The program, called ScanScribe, offers four significant advances. First, it presents a new, intuitive model for maintaining image objects and groups, along with underlying logic for updating these in the course of an editing session. Second, ScanScribe takes advantage of newly developed image processing algorithms to separate foreground markings from a white or light background, and thus can automatically render the background transparent so that image material can be rearranged without occlusion by background pixels. Third, ScanScribe introduces new interface techniques for selecting image objects with a pointing device without resorting to a palette of tool modes. Fourth, ScanScribe presents a platform for exploiting image analysis and recognition methods to make perceptually significant structure readily available to the user. As a research prototype, ScanScribe has proven useful in the work of members of our laboratory, and has been released on a limited basis for user testing and evaluation. KEYWORDS: ScanScribe, rough document, WYPIWYG, perceptual document editing, foreground/background, lattice grouping, bitmap image

In this paper, we describe a technique for dynamically grouping digital ink and audio to support user interaction in freeform note-taking systems. For ink, groups of strokes might correspond to words, lines, or paragraphs of handwritten text. For audio, groups might be a complete spoken phrase or a speaker turn in a conversation. Ink and audio grouping is important for editing operations such as deleting or moving chunks of ink and audio notes. The grouping technique is based on hierarchical agglomerative clustering. This clustering algorithm yields groups of ink or audio in a range of sizes, depending on the level in the hierarchy, and thus provides structure for simple interactive selection and rapid non-linear expansion of a selection. Ink and audio grouping is also important for marking portions of notes for subsequent browsing and retrieval. Integration of the ink and audio clusters provides a flexible way to browse the notes by selecting the ink cluster and playing the correspond...

NoteLook is a client-server system designed and built to support multimedia note taking in meetings with digital video and ink. It is integrated into a conference room equipped with computer controllable video cameras, video conference camera, and a large display rear video projector. The NoteLook client application runs on wireless pen-based notebook computers. Video channels containing images of the room activity and presentation material are transmitted by the NoteLook servers to the clients, and the images can be interactively and automatically incorporated into the note pages. Users can select channels, snap in large background images and sequences of thumbnails, and write freeform ink notes. A smart video source management component enables the capture of high quality images of the presentation material from a variety of sources. For accessing and browsing the notes and recorded video, NoteLook generates Web pages with links from the images and ink strokes correlated to the video.

In this paper we describe a method for indexing and retrieval of multimedia data based on annotation and segmentation. Our goal is the retrieval of segments of audio and video suitable for inclusion in multimedia documents. Annotation refers to the association of text data with particular time locations of the media. Segmentation is the partitioning of continuous media into homogenous regions. Retrieval is performed over segments of the media using the annotations associated with the segments. We present two scenarios that describe how these techniques might be applied. In the first, we describe how excerpts from a video-taped usage study of a new device are located for inclusion in a report on the utility of the device. In the second, we show how sound bites from a recorded meeting are obtained for use in authoring a summary of the meeting. 1.

Theoretical and technological progress has revived the interest in the design of services for the support of co-located human-human communication and collaboration, witnessing the start of several large-scale projects over the last few years. Most of these projects focus on meetings and/or lecture situations. However, usercentred design and evaluation frameworks for co-located communication and collaboration are a major concern. In this paper, we summarise the prevalent approaches towards usercentred design and evaluation, and we develop two different services. In one service, participants in a small-group meeting receive real-time feedback about observable properties of the meeting that are directly related to the social dynamics, such as individual amount of speaking time or eye-gaze patterns. In the other service, teachers in a classroom receive real-time feedback about the activities and attention level of participants in the lecture. We also propose ways to address the different dimensions that are relevant to the design and evaluation of these services (the individual, the social and the organisational dimension), bringing together methods from different disciplines. Categories and Subject Descriptors: H.5.3 [Information interfaces]: Group and organization interfaces – Collaborative computing- Computer-supported

Applications that automatically capture some details of a live experience and provide future access to that experience are increasingly common in the ubiquitous computing community. However, there remains a largely unexplored design space of potential new applications, and very few of the previous systems have been able to evolve in functionality over an extended period of time. To overcome these challenges, we present a distillation of the essential architectural features of an automated capture and access application. We introduce a toolkit, Infrastructure for Capture and Access (INCA) that encourages a simplified model for designing, implementing and evolving capture and access applications. We validate the utility of INCA through three sample applications that show variety within the wider design space and accessibility of the toolkit for others.

This paper describes our experiences in an ongoing collaboration between our research group in Palo Alto and a similar group in Japan. The collaboration is based on semimonthly videoconferences. Due to cultural and language barriers, we found it important to use a variety of communication techniques, ranging from audio and video to text-based chats. During the collaboration, we encountered a number of technical issues that caused communication breakdowns, and have taken various steps to improve the videoconference experience. In addition to improving the sound quality by using better microphone placements, we developed a number of systems that support different aspects of a videoconference. One of the new components is a system for panoramic two-way video communication that digitally combines images from an array of inexpensive video cameras. We also provided a means for the remote display of presentations, made design changes to our conference room, and added support for taking meeting minutes and for reviewing captured videoconferences. These changes have resulted in an improved videoconference experience and in a more effective collaboration with our Japanese colleagues.