The cognitive dimensions framework is a broad-brush evaluation technique for interactive devices and for non-interactive notations. It sets out a small vocabulary of terms designed to capture the cognitively-relevant aspects of structure, and shows how they can be traded off against each other. The purpose of this paper is to propose the framework as an evaluation technique for visual programming environments. We apply it to two commercially-available dataflow languages (with further examples from other systems) and conclude that it is effective and insightful; other HCI-based evaluation techniques focus on different aspects and would make good complements. Insofar as the examples we used are representative, current VPLs are successful in achieving a good `closeness of match', but designers need to consider the `viscosity' (resistance to local change) and the `secondary notation' (possibility of conveying extra meaning by choice of layout, colour, etc.).

Much of software developers’ time is spent understanding unfamiliar code. To better understand how developers gain this understanding and how software development environments might be involved, a study was performed in which developers were given an unfamiliar program and asked to work on two debugging tasks and three enhancement tasks for 70 minutes. The study found that developers interleaved three activities. They began by searching for relevant code both manually and using search tools; however, they based their searches on limited and misrepresentative cues in the code, environment, and executing program, often leading to failed searches. When developers found relevant code, they followed its incoming and outgoing dependencies, often returning to it and navigating its other dependencies; while doing so, however, Eclipse’s navigational tools caused significant overhead. Developers collected code and other information that they believed would be necessary to edit, duplicate, or otherwise refer to later by encoding it in the interactive state of Eclipse’s package explorer, file tabs, and scroll bars. However, developers lost track of relevant code as these interfaces were used for other tasks, and developers were forced to find it again. These issues caused developers to spend, on average, 35 percent of their time performing the mechanics of navigation within and between source files. These observations suggest a new model of program understanding grounded in theories of information foraging and suggest ideas for tools that help developers seek, relate, and collect information in a more effective and explicit manner.

We present results of observing professional maintenance engineers working with industrial code at actual maintenance tasks. Protocol analysis is used to explore how code understanding might differ for small versus large scale code. The experiment confirms that cognition processes work at all levels of abstraction simultaneously as programmers build a mental model of the code. Cognition processes emerged at three levels of aggregation representing lower and higher level strategies of understanding. They show differences in what triggers them and how they achieve their goals. Results are useful for defining core competencies which maintenance engineers need for their work and for documentation and development standards. 1 Introduction During maintenance and software evolution, software engineers must understand code they haven't written for a variety of tasks. Existing cognition models [1, 4, 5, 6, 7, 10] emphasize cognition by what the program does (a functional approach) and how th...

Maintenance frequently consumes more resources than new software development. A major portion of the maintenance effort is spent on the reverse engineering activity of understanding existing software. If we can learn more about how programmers understand code successfully, we can build better tools to support the understanding process. This contributes to higher quality and improved efficiency of maintenance tasks. We present an integrated code comprehension model and our experiences with it in an industrial setting. We use audio-taped, think-aloud reports to investigate how well our integrated code comprehension model works during industrial maintenance activities ranging from code fixes to enhancements, code leverage, and reuse. We analyze the tapes for information needs during maintenance activities and derive tool capabilities accordingly. 1 Introduction Maintenance typically requires more resources than new software development. A significant portion of the maintenance effort in...

The paper presents an overview of the role of concepts in program comprehension. It discusses concept location, in which the implementation of a specific concept is located in the code. This process is very common and precedes a large proportion of code changes. The paper also discusses the process of learning about the domain from the code, which is a prerequisite of code reengineering. The paper notes the similarities and overlaps between program comprehension and human learning.

Despite significant existing empirical work, little is known about the specific kinds of questions programmers ask when evolving a code base. Understanding precisely what information a programmer needs about the code base as they work is key to determining how to better support the activity of programming. The goal of this research is to provide an empirical foundation for tool design based on an exploration of what programmers need to understand about a code base and of how they use tools to discover that information. To this end, we undertook two qualitative studies of programmers performing change tasks to medium to large sized programs. One study involved newcomers working on assigned change tasks to a mediumsized code base. The other study involved industrial programmers working on their own change tasks to code with which they had experience. The focus of our analysis has been on what information a programmer needs to know about a code base while performing a change task and also on how they go about discovering that information. Based on a systematic analysis of the data from these user studies as well as an analysis of the support that current programming tools provide for these activities, this research makes four key contributions: (1) a catalog of 44 types of questions programmers ask, (2) a categorization of those questions into four categories based on the kind and scope of information needed to answer a question, (3) a description of important context for the process of answering questions, and (4) a description of support that is missing from current programming tools.

Software programs, especially legacy programs, are often large, complex and poorly documented. To maintain these programs software engineers require a variety of efficient analytical tools. Some software maintenance tools use visualizations (i.e. graphical views) to communicate information about software systems. Although many software visualization tools exist, the majority of them are not very effective in practice. Part of the problem is that they are designed in an ad hoc manner, with little empirical evaluation. They are often criticized because they try to force programmers to use a specific approach to understanding software rather than supporting their own approaches. The result is that current software visualization tools do not play as big a role in industry as was anticipated by some researchers. The tools that are used are very basic, consisting of mainly text editors and searching features. With increasingly fast computing platforms, there is great potential for the use of...

This paper describes an approach to the feature location problem for distributed systems, that is, to the problem of locating which code components are important in providing a particular feature for an end user. A feature is located by observing system execution and noting time intervals in which it active. Traces of execution in intervals with and without the feature are compared. Earlier experience has shown that this analysis is di#cult because distributed systems often exhibit stochastic behavior and because time intervals are hard to identify with precision.

ABSTRACT Previous studies on program comprehension were carried out largely in the context of procedural languages. Our purpose is to develop and evaluate a cognitive model of object-oriented (OO) program understanding. Our model is based on the van Dijk and Kintsch's model of text understanding (1983). One key aspect of this theoretical approach is the distinction between two kinds of representation the reader might construct from a text: the textbase and the situation model. On the basis of results of an experiment we have conducted, we evaluate the cognitive validity of this distinction in OO program understanding. We examine how the construction of these two representations is differentially affected by the programmer's expertise and how they evolve differentially over time. KEY WORDS mental representation, situation model, program model, object-oriented programming, program comprehension, text comprehension, expertise 1. OBJECTIVES The object-oriented (OO) paradigm is growing fast in popularity, but not enough scientific evidence has been amassed about it. The research that exists is mostly focused on program design and reuse (see for example: Détienne, 1995; Pennington, Lee and Rehder, 1995).

Modern computer systems routinely present information to the user as a combination of text and diagrammatic images, described as "graphical user interfaces". Practitioners and researchers in Human-Computer Interaction (HCI) generally believe that the value of these diagrammatic representations is derived from metaphorical reasoning; they communicate abstract information by depicting a physical situation from which the abstractions can be inferred. This assumption has been prevalent in HCI research for over 20 years, but has seldom been tested experimentally. This thesis analyses the reasons why diagrams are believed to assist with abstract reasoning. It then presents the results of a series of experiments testing the contribution of metaphor to comprehension, problem solving, explanation and memory tasks carried out using a range of different diagrams. The results indicate that explicit metaphors provide surprisingly little benefit for cognitive tasks using diagrams as an external re...