Algorithm visualization (AV) technology graphically illustrates how algorithms work. Despite the intuitive appeal of the technology, it has failed to catch on in mainstream computer science education. Some have attributed this failure to the mixed results of experimental studies designed to substantiate AV technology's educational effectiveness. However, while several integrative reviews of AV technology have appeared, none has focused specifically on the software's effectiveness by analyzing this body of experimental studies as a whole. In order to better understand the effectiveness of AV technology, we present a systematic metastudy of 24 experimental studies. We pursue two separate analyses: an analysis of independent variables, in which we tie each study to a particular guiding learning theory in an attempt to determine which guiding theory has had the most predictive success; and an analysis of dependent variables, which enables us to determine which measurement techniques have been most sensitive to the learning benefits of AV technology. Our most significant finding is that how students use AV technology has a greater impact on effectiveness than what AV technology shows them. Based on our findings, we formulate an agenda for future research into AV effectiveness.

Graphics have been used since ancient times to portray things that are inherently spatiovisual, like maps and building plans. More recently, graphics have been used to portray things that are metaphorically spatiovisual, like graphs and organizational charts. The assumption is that graphics can facilitate comprehension, learning, memory, communication and inference. Assumptions aside, research on static graphics has shown that only carefully designed and appropriate graphics prove to be beneficial for conveying complex systems. Effective graphics conform to the Congruence Principle according to which the content and format of the graphic should correspond to the content and format of the concepts to be conveyed. From this, it follows that animated graphics should be effective in portraying change over time. Yet the research on the efficacy of animated over static graphics is not encouraging. In cases where animated graphics seem superior to static ones, scrutiny reveals lack of equivalence between animated and static graphics in content or procedures; the animated graphics convey more information or involve interactivity. Animations of events may be ineffective because animations violate the second principle of good graphics, the Apprehension Principle, according to which graphics should be accurately perceived and appropriately conceived. Animations are often too complex or too fast to be accurately perceived. Moreover, many continuous events are conceived of as sequences of discrete steps. Judicious use of interactivity may overcome both these disadvantages. Animations may be more effective than comparable static graphics in situations other than conveying complex systems, for example, for real time reorientations in time and space.

Visualization technology can be used to graphically illustrate various concepts in computer science. We argue that such technology, no matter how well it is designed, is of little educational value unless it engages learners in an active learning activity. Drawing on a review of experimental studies of visualization effectiveness, we motivate this position against the backdrop of current attitudes and best practices with respect to visualization use. We suggest a new taxonomy of learner engagement with visualization technology. Grounded in Bloom’s wellrecognized taxonomy of understanding, we suggest metrics for assessing the learning outcomes to which such engagement may lead. Based on these taxonomies of engagement and effectiveness metrics, we present a framework for experimental studies of visualization effectiveness. Interested computer science educators are invited to collaborate with us by carrying out studies within this framework.

A number of prior studies have found that using animation to help teach algorithms had less beneficial effects on learning than hoped. Those results surprise many computer science instructors whose intuition leads them to believe that algorithm animations should assist instruction. This article reports on a study in which animation is utilized in more of a "homework" learning scenario rather than a "final exam" scenario. Our focus is on understanding how learners will utilize animation and other instructional materials in trying to understand a new algorithm, and on gaining insight into how animations can fit into successful learning strategies. The study indicates that students use sophisticated combinations of instructional materials in learning scenarios. In particular, the presence of algorithm animations seems to make a challenging algorithm more accessible and less intimidating, thus leading to enhanced student interaction with the materials and facilitating learning. Keywords:...

QDPLVP RI WKH SUHVHQWDWLRQ PHGLD# 7KH UHVXOWV VXSSRUW D PRGHO#EDVHG DSSURDFK WR WKH GHVLJQ RI PXOWLPRGDO H[SRVLWRU\ SUHVHQWDWLRQV RI G\QDPLF LQIRUPDWLRQ# 7KH FRPSUHKHQVLRQ PRGHO DQG FRUUHVSRQGLQJ GHVLJQ JXLGDQFH VKRXOG DLG GHVLJQHUV LQ EXLOGLQJ LQWHUDFWLYH JUDSKLFDO SUHVHQWDWLRQV WKDW DUH PRUH HIIHFWLYH WKDQ LQWXLWLYH GHVLJQV LQ FRPPXQLFDWLQJ G\QDPLF FRQWHQW# KEYWORDS: interactive graphics, communication of dynamic information, design guidelines and principles; FRPSUHKHQVLRQ process model; empirical studies. 1.# Introduction 0XOWLPRGDO # SUHVHQWDWLRQV# FRQVLVWLQJ# RI# YHUEDO# H[SODQDWLRQV# LOOXVWUDWHG# ZLWK# GLDJUDPV# KDYH ORQJ# EHHQ# XVHG# WR# FRPPXQLFDWH# WHFKQLFDO# LQIRUPDWLRQ# DERXW# WKH# VWUXFWXUH# DQG# G\QDPLF EHKDYLRUV#RI#V\VWHPV##:LWK# WKH#DGYHQW#RI#PXOWLPHGLD## LW# KDV#EHFRPH#SRVVLEOH#WR#PDNH#VXFK SUHVHQWDWLRQV# G\QDPLF# DQG# LQWHUDFWLYH## ,QVWHDG# RI# WKH# WUDGLWLRQDO# FRPELQDWLRQ# RI# WH[W# DQG SLFWXUHV#RQ#D#VWDWLF#PHGLXP#

Computer science educators have traditionally used algorithm visualization (AV) software to create graphical representations of algorithms for use as visual aids in lectures, or as the basis for interactive labs. Typically, such visualizations are high fidelity in the sense that (a) they depict the target algorithm for arbitrary input, and (b) they tend to have the polished look of textbook figures. In contrast, low fidelity visualizations illustrate the target algorithm for a few, carefully chosen input data sets, and tend to have a sketched, unpolished appearance. Drawing on ethnographic field studies of a junior-level algorithms course, we motivate the use of low fidelity AV technology as the basis for an alternative learning paradigm in which students construct their own visualizations, and then present those visualizations to their instructor and peers for feedback and discussion. To explore the design space of low fidelity AV technology, we present SALSA (Spatial ALgorithmic Language for StoryboArding) and ALVIS (ALgorithm VIsualization Storyboarder), a prototype end-user language and system firmly rooted in empirical studies in which students constructed and presented visualizations made out of simple art supplies. Our prototype end-user language and system pioneer a novel technique for programming of visualizations based on spatial relations, and a novel presentation interface that supports human discussions about algorithms by enabling reverse execution and dynamic mark-up and modification. Moreover, the prototype provides an ideal foundation for what we see as the algorithms classroom of the future: the interactive "algorithms studio."

Despite the intuitively compelling adage "a picture is worth a thousand words," attempts over the past decade to use animations to explain algorithms to students have produced disappointing results. In most cases interesting algorithm animations were designed, but no formal, systematic evaluations were conducted. When such evaluations were performed the results were mixed, with compelling evidence for the instructional superiority of algorithm animations failing to emerge. It is in this context that we embarked on a research program to develop educationally effective algorithm visualizations. This program was based on the premise that animations needed to be embedded in a knowledge and context providing hypermedia environment in order to effectively harness their power to enhance learning. This paper describes the architecture of the resulting Hypermedia Algorithm Visualization system HalVis. Four empirical studies with HalVis are described, which demonstrated that the extent of learning exhibited by students who used HalVis was significantly greater than that of students who used means of traditional instruction or a typical algorithm animation.

Adaptive visualization is a technology that can enhance the power of program visualization. The idea of adaptive visualization is to adapt the level of details in a visualization to the level of student knowledge about these constructs. This paper presents an adaptive visualization system, WADEIn, that was developed to explore visualization of expression execution during program execution - a under-explored area in visualization research. WADEIn has been designed as a component of our distributed Web-based adaptive educational system KnowledgeTree, however it also can be used as a standalone educational tool. The system has been pilot-tested in the context of a real university course with 40 students and is available on the Web for public use.

We present research leading toward an understanding of the optimal audio-visual representation for illustrating concepts for illiterate and semi-literate users of computers. In our user study, which to our knowledge is the first of its kind, we presented to 200 illiterate subjects each of 13 different health symptoms in one representation randomly selected among the following ten: text, static drawings, static photographs, hand-drawn animations, and video, each with and without voice annotation. The goal was to see how comprehensible these representation types were for an illiterate audience. We used a methodology for generating each of the representations tested in a way that fairly stacks one representational type against the others. Our main results are that (1) voice annotation generally helps in speed of comprehension, but bimodal audio-visual information can be confusing for the target population; (2) richer information is not necessarily better understood overall; (3) the relative value of dynamic imagery versus static imagery depends on various factors. Analysis of these statistically significant results and additional detailed results are also provided.

WWW home page:http://www.cs.joensuu.fi/˜snevalai/ Abstract. The empirical evaluation of program visualisation has been based mostly on observations of long-term effects of the program visualisation tools, while possible short-term effects of the visualisations and their relation to the long-term effects have been elided. In order to study short-term effects of visualisation of variables in a context where the long-term effects are already known, we conducted a controlled experiment, in which we investigated how a person targets her visual attention and what kind of a mental model she constructs, when variables are presented either textually or graphically. The results indicate clear differences in the targeting of visual attention between the visualisation tools: With the graphical tool, the participants targeted their visual attention to variables much more than with the textual tool. With the graphical tool, the increase of visual attention to variables increased the proportion of high-level information in program summaries and decreased the proportion of low-level code-related information. 1