Although both statistical methods and visualizations have been used by network analysts, exploratory data analysis remains a challenge. We propose that a tight integration of these technologies in an interactive exploratory tool could dramatically speed insight development. To test the power of this integrated approach, we created a novel social network analysis tool, SocialAction, and conducted four long-term case studies with domain experts, each working on unique data sets with unique problems. The structured replicated case studies show that the integrated approach in SocialAction led to significant discoveries by a political analyst, a bibliometrician, a healthcare consultant, and a counter-terrorism researcher. Our contributions demonstrate that the tight integration of statistics and visualizations improves exploratory data analysis, and that our evaluation methodology for long-term case studies captures the research strategies of data analysts.

Even though “providing insight ” has been considered one of the main purposes of information visualization (InfoVis), we feel that insight is still a not-well-understood concept in this context. Inspired by research in sensemaking, we realized the importance of the procedural aspects in understanding insight. Thus, rather than asking “What is insight? ” we instead focus on “How do people gain insights? ” In an effort to better understand and characterize insight, we reviewed previous literature in InfoVis, seeking other researchers ’ comments and views on this concept. We found that: 1) Insights are often regarded as end results of using InfoVis and the procedures to gain insight have been largely veiled; 2) Four largely distinctive processes of gaining insight (Provide Overview, Adjust, Detect Pattern, and Match Mental Model) have been discussed in the InfoVis literature; and 3) These different processes provide some hints to understand the procedures in which insight can be gained from InfoVis. We hope that our findings help researchers and practitioners evaluate InfoVis systems and technologies in a more insight-oriented way. Categories and Subject Descriptors

Abstract. Researchers and users of Information Visualization are convinced that it has value. This value can easily be communicated to others in a face-to-face setting, such that this value is experienced in practice. To convince broader audiences, and also, to understand the intrinsic qualities of visualization is more difficult, however. In this paper we consider information visualization from different points of view, and gather arguments to explain the value of our field. 1 Problems and Challenges This paper provides a discussion of issues surrounding the value of Information Visualization (InfoVis). The very existence of the paper should alert the reader that challenges do exist in both recognizing and communicating the field’s value. After all, if the value would be clear and undisputed, there would be no need to write the paper! Unfortunately, the current situation is far from that. By its very focus and purpose, InfoVis is a discipline that makes the recognition of value extremely difficult, a point that will be expanded below. Why is showing value important? Well, today’s research environment places great importance on evaluation involving quantifiable metrics that can be assessed and judged with clarity and accuracy. Organizations sponsoring research and corporations that serve to benefit from it want to know that the monetary investments they make are being well-spent. Researchers are being challenged

Abstract — In Human-Computer studies and visual analytics, the majority of the evaluation methods applied, collect data at a single point in time, in form of cross-sectional data. In several studies [e.g. 8] numerous visualization tools were evaluated in controlled experiments. Although the experiments discovered valuable findings, certain drawbacks of the research method were expressed. The time constraints of one-time experiments reduce the amount of training which can be given to the participants. Furthermore, when the studies tried to measure the insight derived from the visualization tools the time constraints didn’t allow observing how these insights develop over time or their interdependency. Further problems of cross-sectional studies are well known, like the selection of appropriate tasks, the mostly extrinsic motivation of the participants, the influence of a laboratory environment compared to a realistic work setting and whether a visualization tool does meet the work requirements in the long run. In this position paper we argue for applying longitudinal research methods in human-computer studies as an extension to cross-sectional studies and present a first approach towards a methodological research framework. We suggest a set of research questions and performance measures that would be benefical for extending cross-sectional studies with longitudinal ones. We also describe in two case studies, in which only cross-sectional research methods were used, how they can improved by longitudinal methods. Index Terms—longitudinal, long-term, methodology, research framework, experiment, visual analytics, evaluation 1

Social Network Analysis (SNA) has evolved as a popular, standard method for modeling meaningful, often hidden structural relationships in communities. Existing SNA tools often involve extensive pre-processing or intensive programming skills that can challenge practitioners and students alike. NodeXL, an open-source template for Microsoft Excel, integrates a library of common network metrics and graph layout algorithms within the familiar spreadsheet format, offering a potentially low-barrierto-entry framework for teaching and learning SNA. We present the preliminary findings of 2 user studies of 21 graduate students who engaged in SNA using NodeXL. The majority of students, while information professionals, had little technical background or experience with SNA techniques. Six of the participants had more technical backgrounds and were chosen specifically for their experience with graph drawing and information visualization. Our primary objectives were (1) to evaluate NodeXL as an SNA tool for a broad base of users and (2) to explore methods for teaching SNA. Our complementary dual case-study format demonstrates the usability of NodeXL for a diverse set of users, and significantly, the power of a tightly integrated metrics/visualization tool to spark insight and facilitate sensemaking for students of SNA. I.

Despite the long history and consistent use of quantitative empirical methods to evaluate information visualization techniques and systems, our understanding of interface use remains incomplete. While there are inherent limitations to the method, such as the choice of task and data, we believe the utility of study results can be enhanced if they were amenable to meta-analysis. Based on our experience in extracting design guidelines from existing quantitative studies, we recommend improvements to both study design and reporting to promote meta-analysis: (1) Use comparable interfaces in terms of visual elements, information content and amount displayed, levels of data organization displayed, and interaction complexity; (2) Capture usage patterns in addition to overall performance measurements to better identify design tradeoffs; (3) Isolate and study interface factors instead of overall interface performance; and (4) Report more study details, either within the publications, or as supplementary materials.

Pathway diagrams, similar to the graph diagrams using a node-link representation, are used by biologists to represent complex interactions at the molecular level in living cells. The recent shift towards data-intensive bioinformatics and systems-level science has created a strong need for advanced pathway visualization tools that support exploratory data analysis. User studies suggest that an important requirement for biologists is the need to associate microarray data to pathway diagrams. A design space for visualization tools that allow analysis of microarray data in pathway context was identified for a systematic evaluation of the visualization alternatives. The design space is divided into two dimensions. Dimension 1 is based on the method used to overlay data attributes onto pathway nodes. The three possible approaches are: overlay of data on pathway nodes one data attribute at a time by manipulating a visual property (e.g. color) of the node, along with sliders or some such mechanism to animate the pathway for other timepoints. In another approach data from all the attributes in data can be overlaid simultaneously by embedding small charts (e.g.,

Abstract—We take a new, scenario based look at evaluation in information visualization. Our seven scenarios, evaluating visual data analysis and reasoning, evaluating user performance, evaluating user experience, evaluating environments and work practices, evaluating communication through visualization, automated evaluation of visualizations, and evaluating collaborative data analysis were derived through an extensive literature review of over 800 visualization publications. These scenarios are described through their goals, the types of questions they embody and illustrated through example studies. Through this broad survey and the distillation of these scenarios we make two contributions. One, we encapsulate the current practices in the information visualization research community and, two, we provide a different approach to reaching decisions about what might be the most effective evaluation of a given information visualization. For example, if the research goals or evaluative questions are known they can be used to map to specific scenarios, where practical existing examples can be considered for effective evaluation approaches. Index Terms—Information visualization, evaluation 1

Acknowlegement: This work was supported by NIH 5R24HD050846-02 Integrated molecular core for rehabilitation medicine, and NIH 1P30HD40677-01 (MRDDRC Genetics Core). We also thank the FMS study group, especially Joseph Devaney and Eric Hoffman, for providing genotype data. Corresponding Author’s Contact Information:

The IEEE Visual Analytics Science and Technology (VAST) Symposium has held a contest each year since its inception in 2006. These events are designed to provide visual analytics researchers and developers with analytic challenges similar to those encountered by professional information analysts. The VAST contest has had an extended life outside of the symposium, however, as materials are being used in universities and other educational settings, either to help teachers of visual analyticsrelated classes or for student projects. We describe how we develop VAST contest datasets that results in products that can be used in different settings and review some specific examples of the adoption of the VAST contest materials in the classroom. The examples are drawn from graduate and undergraduate courses at Virginia Tech and from the Visual Analytics “Summer Camp ” run by the National Visualization and Analytics Center in 2008. We finish with a brief discussion on evaluation metrics for education.