We present three novel tools for creating data graphics: (1) SageBrush, for assembling graphics from primitive objects like bars, lines and axes, (2) SageBook, for browsing previously created graphics relevant to current needs, and (3) SAGE, a knowledge-based presentation system that automatically designs graphics and also interprets a user's specifications conveyed with the other tools. The combination of these tools supports two complementary processes in a single environment: design as a constructive process of selecting and arranging graphical elements, and design as a process of browsing and customizing previous cases. SAGE enhances userdirected design by completing partial specifications, by retrieving previously created graphics based on their appearance and data content, by creating the novel displays that users specify, and by designing alternatives when users request them. Our approach was to propose interfaces employing styles of interaction that appear to support graphic d...

When working with large data sets, users perform three primary types of activities: data manipulation, data analysis, and data visualization. The data manipulation process involves the selection and transformation of data prior to viewing. This paper addresses user goals for this process and the interactive interface mechanisms that support them. We consider three classes of data manipulation goals: controlling the scope (selecting the desired portion of the data), selecting the focus of attention (concentrating on the attributes of data that are relevant to current analysis), and choosing the level of detail (creating and decomposing aggregates of data). We use this classification to evaluate the functionality of existing data exploration interface techniques. Based on these results, we have expanded an interface mechanism called the Aggregate Manipulator (AM) and combined it with Dynamic Query (DQ) to provide complete coverage of the data manipulation goals. We use real estate sales data to demonstrate how the AM and DQ synergistically function in our interface.

HUMANOID is a user interface design tool that lets designers express abstract conceptualizations of an interface in an executable form, allowing designers to experiment with scenarios and dialogues even before the application model is completely worked out. Three properties of the HUMANOID approach allow it to do so: a modularization of design issues into independent dimensions, support for multiple levels of specificity in mapping application models to user interface constructs, and mechanisms for constructing executable default user interface implementations from whatever level of specificity has been provided by the designer. KEYWORDS: Design Processes, Development Tools and Methods, User Interface Management Systems, Rapid Prototyping, Interface Design Representation, Dialogue Specification. INTRODUCTION Interface design really begins much earlier than current tools recognize. Long before a designer is ready to experiment with presentation issues like the layout of widgets chosen...

New user interface challenges are arising because people need to explore and perform many diverse tasks involving large quantities of abstract information. Visualizing information is one approach to these challenges. But visualization must involve much more than just enabling people to "see" information. People must also manipulate it to focus on what is relevant and reorganize it to create new information. They must also communicate and share information in collaborative settings and act directly to perform their tasks based on this information. These goals suggest the need for information visualization workspaces with new interaction approaches. We present several systems - Visage, SAGE and SDM - that comprise such a workspace and a suite of user interface techniques for creating and manipulating integrative visualizations. Our work in this area revealed the need for interfaces that enable people to communicate with systems in multiple complementary ways. We discuss four dimensions f...

We address one of the problems at the heart of automated multimedia presentation production and interpretation. The media allocation problem can be stated as follows: how does the producer of a presentation determine which information to allocate to which medium, and how does a perceiver recognize the function of each part as displayed in the presentation and integrate them into a coherent whole? What knowledge is used, and what processes? We describe the four major types of knowledge that play a role in the allocation problem as well as interdependencies that hold among them. We discuss two formalisms that can be used to represent this knowledge and, using examples, describe the kinds of processing required for the media allocation problem. wip comet aimi 1 The General Problem of Presentations using Multiple Media When communicating, people almost always employ multiple modalities. Even natural language, which is after all the most powerful representational medium developed by hum...

Graphical presentations can be used to communicate information in relational data sets succinctly and effectively. However, novel graphical presentations about numerous attributes and their relationships are often difficult to understand completely until explained. Automatically generated graphical presentations must therefore either be limited to simple, conventional ones, or risk incomprehensibility. One way of alleviating this problem is to design graphical presentation systems that can work in conjunction with a natural language generator to produce "explanatory captions." This paper presents three strategies for generating explanatory captions to accompany information graphics based on: (1) a representation of the structure of the graphical presentation (2) a framework for identifyingthe perceptual complexity of graphical elements, and (3) the structure of the data expressed in the graphic. We describe an implemented system and illustrate how it is used to generate explanatory cap...

... This paper presents a system to do so. It uses a text planner to determine the content and structure of the captions based on: (1) a representation of the structure of the graphical presentation and its mapping to the data it depicts, (2) a framework for identifying the perceptual complexity of graphical elements, and (3) the structure of the data expressed in the graphic. The output of the planner is further processed regarding issues such as ordering, aggregation, centering, generating referring expressions and lexical choice. We discuss the architecture of our system and its strengths and limitations. Our implementation is currently limited to 2-D charts and maps, but, except for lexical information, it is completely domain independent. We illustrate our discussion with figures and generated captions about housing sales in Pittsburgh.

We address the problem of realizing communicative plans in graphics. Our approach calls for mapping communicative goals to conceptual tasks and then using task-based graphic design for selecting graphical techniques. In this paper, we present the mapping rules in several dimensions: data aggregation and selection, task synthesis, and task aggregation. Those rules have been incorporated in AutoBrief, a research system for multimedia explanation. Keywords Data graphic design, communicative plans, conceptual tasks 1. INTRODUCTION Visualizations are used in narratives, arguments, explanations and other communicative genres to succinctly convey complex relations or to organize a large amount of information. Such presentations are planned to achieve communicative goals (e.g., the user believes that insufficient airport capacity is the cause for some late cargo). Communicative planning involves reasoning about communicative goals, user's beliefs, and information about the domain, in order t...

This paper describes a media-independent knowledge representation scheme, or content language, for describing the content of communicative goals and actions. The language is used within an intelligent system for automatically generating integrated text and information graphics presentations about complex, quantitative information. The language is designed to satisfy four requirements: to represent information about complex quantitative relations and aggregate properties; compositionality; to represent certain pragmatic distinctions needed for satisfying communicative goals; and to be usable as input by the media-specific generators in our system. 1

Abstract—The design and evaluation of most current information visualization systems descend from an emphasis on a user’s ability to “unpack ” the representations of data of interest and operate on them independently. Too often, successful decision-making and analysis are more a matter of serendipity and user experience than of intentional design and specific support for such tasks; although humans have considerable abilities in analyzing relationships from data, the utility of visualizations remains relatively variable across users, data sets, and domains. In this paper, we discuss the notion of analytic gaps, which represent obstacles faced by visualizations in facilitating higher-level analytic tasks, such as decision-making and learning. We discuss support for bridging these gaps, propose a framework for the design and evaluation of information visualization systems, and demonstrate its use. Index Terms—Information visualization, visualization techniques and methodologies, theory and methods. 1