Toolglass ™ widgets are new user interface tools that can appear, as though on a transparent sheet of glass, between an application and a traditional cursor. They can be positioned with one hand while the other positions the cursor. The widgets provide a rich and concise vocabulary for operating on application objects. These widgets may incorporate visual filters, called Magic Lens™ filters, that modify the presentation of application objects to reveal hidden information, to enhance data of interest, or to suppress distracting information. Together, these tools form a see-through interface that offers many advantages over traditional controls. They provide a new style of interaction that better exploits the user’s everyday skills. They can reduce steps, cursor motion, and errors. Many widgets can be provided in a user interface, by designers and by users, without requiring dedicated screen space. In addition, lenses provide rich context-dependent feedback and the ability to view details and context simultaneously. Our widgets and lenses can be combined to form operation and viewing macros, and can be used over multiple applications.

This paper presents a new afgorithm for smoothly blending between two 2-D polygonal shapes. The algorithm is based on a physical model wherein one of the shapes is considered to be constructed of wire, and a solution is found whereby the first shape can be bent and/or stretched into the second shape with a minimum amount of work. The resulting solution tends to associate regions on the two shapes which look alike. If the two polYgons have m and n vertices respectively, the afgorithm is O(mn). The algorithm avoids local shape inversions in whkh intermediate

One of the problems with conventional UIMSs is that very often there is no graphical way to specify interfaces. This paper describes OPUS, the user interface editor of the Penguims UIMS. This system allows the presentation component of graphical user interfaces to be specified interactively in a graphical notation without explicit programming. The Penguims UIMS supports an underlying model of computation based loosely on spreadsheets. In particular, it supports incremental computations based on a system of equations (one-way constraints) over a set of named values (spreadsheet cells). These equations are used to provide immediate feedback at all levels of the interface. They are used to incrementally determine the position and dynamic appearance of the individual interactor objects that make up the interface. They are also used to connect the presentation directly to underlying application data thereby supporting semantic feedback. The OPUS user interface editor employs a special graphical notation for specifying the presentation component of a user interface. This notation allows the power of the underlying computational model to be expressed simply and quickly. The resulting presentations are very flexible in nature. They can automatically respond to changes in the size and position of display objects and can directly support derivation of their appearance from application data objects.

This paper discusses aspects of multimedia document bases and how access to documents held on a computer-based system can be achieved; in particular the current access methods of hypermedia and free text information retrieval are discussed. Browsing-based hypermedia systems provide ease of use for novice users and equal access to any media; however, they typically perform poorly with very large document bases. In contrast query-based free text retrieval systems are, typically, designed to work with very large document bases, but have very poor multimedia capabilities. This paper then presents a hybrid between these two traditional fields of information retrieval together with a technique for using contextual information to provide access, through query, to documents which cannot be accessed by content (e.g. images). Two experiments are then presented which were carried out to test this approach. Finally, the paper gives a brief discussion of a prototype implementation, which provides access to mixed media information by query or browsing, and user-interface issues are discussed. * A version of this paper was presented at the RIAO91 conference in Barcelona (Dunlop and Van Rijsbergen 1991). This paper completely supersedes the conference paper in the development and testing of the model, however, the conference paper gives more details on the prototype application and wider issues described at the end of this paper. + May be reached by e-mail to mark@dcs.glasgow.ac.uk. HYPERMEDIA & FREE TEXT RETRIEVAL 2

Abstract — While there have been advances in visualization systems, particularly in multi-view visualizations and visual exploration, the process of building visualizations remains a major bottleneck in data exploration. We show that provenance metadata collected during the creation of pipelines can be reused to suggest similar content in related visualizations and guide semi-automated changes. We introduce the idea of query-by-example in the context of an ensemble of visualizations, and the use of analogies as first-class operations in a system to guide scalable interactions. We describe an implementation of these techniques in VisTrails, a publicly-available, open-source system. Index Terms—visualization systems, query-by-example, analogy 1

Hand drawn diagrams are essential tools for thinking and communicating in the early phases of design, yet computer based drawing tools support diagramming and sketching only poorly. Key components of computational support for early design include recognition, interpretation, and management of diagrams The paper describes the motivation for, implementation of, and initial experience with the "computer as cocktail napkin " project, a design environment based on diagrams. It explains low level recognition of glyphs, construction of higher-level recognizers, and routines for managing diagrams in the cocktail napkin prototype. 1.

We present the concept of an editable graphical history that allows the user to review and modify the actions performed with a graphical user interface. Using a pictorial metaphor borrowed from comic strips, an editable graphical history consists of a series of panels that depict in chronological order the important events in the history of a user's session. We discuss the visual language used in editable graphical histories, and describe Chimera, a graphical editor that generates these histories automatically. The user may scroll through the sequence of panels, reviewing actions at different levels of detail, and selectively undoing, modifying, and redoing previous actions. Chimera's editable graphical histories are constructed from parts of the editor window, the editor control panel, and the editor's pop up menus. Panels indicate both the objects that are modified and the actions performed on them. We describe the heuristics used to determine the objects depicted in each panel, the ...

Graphical editing, like many applications facilitated by computers, often involves repetitive tasks. To reduce repetition, programmers can write procedures to automate these tasks, however most users do not know how to program,

In Programming by Example [PBE, also sometimes called "Programming by Demonstration"] systems, the system records actions performed by a user in the interface, and produces a generalized program that can be used later in analogous examples. A key issue is how to describe the actions and objects selected by the user, which determines what kind of generalizations will be possible. When the user selects a graphical object on the screen, most PBE systems describe the object using properties of the underlying application data. For example, if the user selects a link on a web page, the PBE system might represent the selection based on the link's HTML properties. In this article, we explore a different, and radical, approach-- using visual properties of the interaction elements themselves, such as size, shape, color, and appearance of graphical objects-- to describe user intentions. Only recently has the speed of image processing made feasible real-time analysis of screen images by a PBE system. We have not yet fully realized the goal of a complete PBE system using visual generalization, but we feel the approach is important enough to warrant presenting the idea.

Writing scripts (often called "macros") can be helpful for automating repetitive tasks. Scripting facilities for text editors like Emacs and Microsoft Word have been widely used and available. However, for graphical applications, scripting has been tried many times but has never been successful. This is mainly due to the data description problem of determining how to generalize the particular objects selected at demonstration time. Previous systems have mostly tried to solve this using inferencing, but this has a number of problems, including guessing wrong and providing appropriate feedback and control to users. Therefore, the Topaz framework does not use inferencing and instead allows the user to specify how the appropriate objects should be found. This is achieved by recording changes to which objects are selected and searches for objects, so that scripts can be written with respect to the selected object, in the same way as Emacs keyboard macros. Furthermore, all values can be expl...