High-throughput experiments such as gene expression microarrays in the life sciences result in large datasets. In response, a wide variety of visualization tools have been created to facilitate data analysis. Biologists often face a dilemma in choosing the best tool for their situation. The tool that works best for one biologist may not work well for another due to differences in the type of insight they seek from their data. A primary purpose of a visualization tool is to provide domain-relevant insight into the data. Ideally, any user wants maximum information in the least possible time. In this paper we identify several distinct characteristics of insight that enable us to recognize and quantify it. Based on this, we empirically evaluate five popular microarray visualization tools. Our conclusions can guide biologists in selecting the best tool for their data, and computer scientists in developing and evaluating visualizations.

Abstract—High-throughput experiments, such as gene expression microarrays in the life sciences, result in very large data sets. In response, a wide variety of visualization tools have been created to facilitate data analysis. A primary purpose of these tools is to provide biologically relevant insight into the data. Typically, visualizations are evaluated in controlled studies that measure user performance on predetermined tasks or using heuristics and expert reviews. To evaluate and rank bioinformatics visualizations based on real-world data analysis scenarios, we developed a more relevant evaluation method that focuses on data insight. This paper presents several characteristics of insight that enabled us to recognize and quantify it in open-ended user tests. Using these characteristics, we evaluated five microarray visualization tools on the amount and types of insight they provide and the time it takes to acquire it. The results of the study guide biologists in selecting a visualization tool based on the type of their microarray data, visualization designers on the key role user interaction techniques, and evaluators on a new approach for evaluating the effectiveness of visualizations for providing insight. Though we used the method to analyze bioinformatics visualizations, it can be applied to other domains. Index Terms—Evaluation/methodology, graphical user interfaces (GUI), information visualization, visualization systems and software, visualization techniques and methodologies. æ 1

Although both statistical methods and visualizations have been used by network analysts, exploratory data analysis remains a challenge. We propose that a tight integration of these technologies in an interactive exploratory tool could dramatically speed insight development. To test the power of this integrated approach, we created a novel social network analysis tool, SocialAction, and conducted four long-term case studies with domain experts, each working on unique data sets with unique problems. The structured replicated case studies show that the integrated approach in SocialAction led to significant discoveries by a political analyst, a bibliometrician, a healthcare consultant, and a counter-terrorism researcher. Our contributions demonstrate that the tight integration of statistics and visualizations improves exploratory data analysis, and that our evaluation methodology for long-term case studies captures the research strategies of data analysts.

Abstract—Visualization tools are typically evaluated in controlled studies that observe the short-term usage of these tools by participants on preselected data sets and benchmark tasks. Though such studies provide useful suggestions, they miss the long-term usage of the tools. A longitudinal study of a bioinformatics data set analysis is reported here. The main focus of this work is to capture the entire analysis process that an analyst goes through from a raw data set to the insights sought from the data. The study provides interesting observations about the use of visual representations and interaction mechanisms provided by the tools, and also about the process of insight generation in general. This deepens our understanding of visual analytics, guides visualization developers in creating more effective visualization tools in terms of user requirements, and guides evaluators in designing future studies that are more representative of insights sought by users from their data sets. Index Terms—Evaluation/methodology, Graphical User Interface (GUI), information visualization, visualization systems and software, visualization and methodologies. 1

Knowledge discovery in high dimensional data is a challenging enterprise, but new visual analytic tools appear to offer users remarkable powers if they are ready to learn new concepts and interfaces. Our 3-year effort to develop versions of the Hierarchical Clustering Explorer (HCE) began with building an interactive tool for exploring clustering results. It expanded, based on user needs, to include other potent analytic and visualization tools for multivariate data, especially the rank-by-feature framework. Our own successes using HCE provided some testimonial evidence of its utility, but we felt it necessary to get beyond our subjective impressions. This paper presents an evaluation of the Hierarchical Clustering Explorer (HCE) using three case studies and an email user survey (n=57) to focus on skill acquisition with the novel concepts and interface for the rank-by-feature framework. Knowledgeable and motivated users in diverse fields provided multiple perspectives that refined our understanding of strengths and weaknesses. A user survey confirmed the benefits of HCE, but gave less guidance about improvements. Both evaluations suggested improved training methods.

People often need to quickly access or maintain awareness of secondary information while busy with other primary tasks. Information visualizations provide rapid, effective access to information, but are generally designed to be examined by users as the primary focus of their attention. The goal of this research is to discover how to design information visualizations intended for the periphery and to understand how quickly and effectively people can interpret information visualizations while they are busily performing other tasks. We evaluated how several factors of a visualization (visual density, presence time, and secondary task type) impact people's abilities to continue with a primary task and to complete secondary tasks related to the visualization. Our results suggest that, with relaxed time pressure, reduced visual information density and a single well-defined secondary task, people can effectively interpret visualizations with minimal distraction to their primary task.

Gravi++ is a visualization method that is designed to support psychotherapists in their work with anorectic girls. During the therapy complex and time dependent data have to be analyzed. Statistical methods cannot be used in this context, and visualizations seem to be a viable alternative. Gravi++ is based on a spring metaphor. It can represent time dependent data easily. There is still too little systematic and empirically validated knowledge to support the design of such information visualizations. Therefore, extensive evaluation is necessary. The evaluation process is composed of two steps – a usability study and the evaluation of the Gravi++ method as such. The following paper describes the usability study. Methods used were usability inspection / guideline review, heuristic evaluation and focus groups. Heuristic evaluation was a very valuable method for identifying usability problems. Focus groups did not yield very much additional factual knowledge but gave important insights about the subjective importance of usability problems.

This paper reports the findings of an experiment conducted to determine whether graphical or textual representations of a simulated load monitor are more effective at communicating information in a dual-task environment. Results include guidelines for design tradeoffs based on significant differences in display facilitation of information awareness, communication, and introduction of distraction. This research is critical in developing a framework for dual-task evaluation that should guide the design and use of systems that require a division of user attention. 1.

As information visualization tools are used to visualize datasets of increasing size, there is a growing need for techniques that facilitate efficient navigation. Pan and zoom navigation enables users to display areas of interest at different resolutions. Focus+context techniques aim to overcome the drawbacks of pan and zoom by dynamically integrating areas of interest and context regions. To date, empirical comparisons of these two navigation paradigms have been limited in scope and inconclusive. In two controlled studies, we evaluated navigation techniques representa-tive of the pan and zoom and focus+context approaches. The particular fo-cus+context technique examined was rubber sheet navigation, implemented in a way that afforded a set of navigation actions similar to pan and zoom navi-gation. The two techniques were used by 40 subjects in each study to perform a navigation-intensive task in a large tree dataset. Study 1 investigated the effect of the amount of screen real estate devoted to context regions for each navigation technique. Performance with both techniques was not significantly affected by this factor, but was influenced by technique-specific strategies de-veloped by subjects. Study 2 compared the performance of the two techniques. Pan and zoom navigation was found to be faster than rubber sheet navigation and was rated by subjects as easier and less mentally demanding. We discuss the implications of these results, including the relationship between navigation technique, task, and user strategy, and propose directions for future work. ii

This paper reports the findings of a humancomputer interaction (HCI) experiment, conducted to determine whether graphical or textual representations of a simulated load monitor are more effective at communicating notification information in a secondary display. We establish guidelines for design tradeoffs based on significant differences in display facilitation of information monitoring, awareness, and introduction of distraction. These findings result from an experiment in which subjects browsed through information pages searching for answers to questions while simultaneously monitoring information in the load monitor. This research is critical in developing a framework for secondary display evaluation that should guide the design and use of visual notification systems requiring a division of user attention.