The cognitive dimensions framework is a broad-brush evaluation technique for interactive devices and for non-interactive notations. It sets out a small vocabulary of terms designed to capture the cognitively-relevant aspects of structure, and shows how they can be traded off against each other. The purpose of this paper is to propose the framework as an evaluation technique for visual programming environments. We apply it to two commercially-available dataflow languages (with further examples from other systems) and conclude that it is effective and insightful; other HCI-based evaluation techniques focus on different aspects and would make good complements. Insofar as the examples we used are representative, current VPLs are successful in achieving a good `closeness of match', but designers need to consider the `viscosity' (resistance to local change) and the `secondary notation' (possibility of conveying extra meaning by choice of layout, colour, etc.).

Software engineering research has focused mainly on software construction and has neglected software maintenance and evolution. Proposed is a shift in research from synthesis to analysis. Reverse engineering is introduced as a possible solution to program understanding and software analysis. Presented is reverse engineering technology developed as part of the Rigi project. The Rigi approach involves the identification of software artifacts in the subject system and the aggregation of these artifacts to form more abstract architectural models. Reported are some analyses on the source code of SQL/DS, performed by the authors while visiting the Program Understanding project at the IBM Centre for Advanced Studies in Toronto.

In this paper we review the literature relating to the psychological/educational study of programming. We identify general trends comparing novice and expert programmers, programming knowledge and strategies, program generation and comprehension, and objectoriented versus procedural programming. (We do not cover research relating specifically to other programming styles.) The main focus of the review is on novice programming and topics relating to novice teaching and learning. Various problems experienced by novices are identified, including issues relating to basic program design, to algorithmic complexity in certain language features, to the ‘‘fragility’ ’ of novice knowledge, and so on. We summarise this material and suggest some practical implications for teachers. We suggest that a key issue that emerges is the distinction between effective and ineffective novices. What characterises effective novices? Is it possible to identify the specific deficits of ineffective novices and help them to become effective learners of programming? 1.

Much of software developers’ time is spent understanding unfamiliar code. To better understand how developers gain this understanding and how software development environments might be involved, a study was performed in which developers were given an unfamiliar program and asked to work on two debugging tasks and three enhancement tasks for 70 minutes. The study found that developers interleaved three activities. They began by searching for relevant code both manually and using search tools; however, they based their searches on limited and misrepresentative cues in the code, environment, and executing program, often leading to failed searches. When developers found relevant code, they followed its incoming and outgoing dependencies, often returning to it and navigating its other dependencies; while doing so, however, Eclipse’s navigational tools caused significant overhead. Developers collected code and other information that they believed would be necessary to edit, duplicate, or otherwise refer to later by encoding it in the interactive state of Eclipse’s package explorer, file tabs, and scroll bars. However, developers lost track of relevant code as these interfaces were used for other tasks, and developers were forced to find it again. These issues caused developers to spend, on average, 35 percent of their time performing the mechanics of navigation within and between source files. These observations suggest a new model of program understanding grounded in theories of information foraging and suggest ideas for tools that help developers seek, relate, and collect information in a more effective and explicit manner.

A major portion of the maintenance effort is spent understanding existing software. We present an integrated code comprehension model and our experiences with it in an industrial setting. We use audio-taped, think-aloud reports to investigate how well our integrated code comprehension model works during industrial maintenance activities ranging from code fixes to enhancements, code leverage, and reuse. We analyze the tapes for information needs during maintenance activities and derive tool capabilities accordingly. 1 Introduction A significant portion of the maintenance effort involves the code comprehension process. Typical tasks that require understanding include troubleshooting, code leverage (reuse with modification), and program enhancement. The first step in satisfying a maintenance engineer 's information needs is to define a model of how programmers understand code. The literature [1, 2, 8, ?, 13, 15, 16, 17], provides two approaches to comprehension: cognitive models that e...

We present results of observing professional maintenance engineers working with industrial code at actual maintenance tasks. Protocol analysis is used to explore how code understanding might differ for small versus large scale code. The experiment confirms that cognition processes work at all levels of abstraction simultaneously as programmers build a mental model of the code. Cognition processes emerged at three levels of aggregation representing lower and higher level strategies of understanding. They show differences in what triggers them and how they achieve their goals. Results are useful for defining core competencies which maintenance engineers need for their work and for documentation and development standards. 1 Introduction During maintenance and software evolution, software engineers must understand code they haven't written for a variety of tasks. Existing cognition models [1, 4, 5, 6, 7, 10] emphasize cognition by what the program does (a functional approach) and how th...

Maintenance frequently consumes more resources than new software development. A major portion of the maintenance effort is spent on the reverse engineering activity of understanding existing software. If we can learn more about how programmers understand code successfully, we can build better tools to support the understanding process. This contributes to higher quality and improved efficiency of maintenance tasks. We present an integrated code comprehension model and our experiences with it in an industrial setting. We use audio-taped, think-aloud reports to investigate how well our integrated code comprehension model works during industrial maintenance activities ranging from code fixes to enhancements, code leverage, and reuse. We analyze the tapes for information needs during maintenance activities and derive tool capabilities accordingly. 1 Introduction Maintenance typically requires more resources than new software development. A significant portion of the maintenance effort in...

We identify and analyze basic principles which necessarily underlie software measures research. In the prevailing paradigm for the validation of software measures there is a fundamental assumption that the sets of measured documents are ordered, and that measures should report these orders. We describe mathematically the nature of such orders. Consideration of these orders suggests a hierarchy of software document measures, a methodology for developing new measures, and a general approach to the analytical evaluation of measures. We also point out the importance of units for any type of measurement and stress the perils of equating document structure complexity and psychological complexity. Keywords: software measures, abstractions of software documents, software structure, analytical evaluations of measures 1 Introduction This paper presents some underlying principles for software measures research. By "software measures" we mean measures which are obtainable directly from software d...

Despite significant existing empirical work, little is known about the specific kinds of questions programmers ask when evolving a code base. Understanding precisely what information a programmer needs about the code base as they work is key to determining how to better support the activity of programming. The goal of this research is to provide an empirical foundation for tool design based on an exploration of what programmers need to understand about a code base and of how they use tools to discover that information. To this end, we undertook two qualitative studies of programmers performing change tasks to medium to large sized programs. One study involved newcomers working on assigned change tasks to a mediumsized code base. The other study involved industrial programmers working on their own change tasks to code with which they had experience. The focus of our analysis has been on what information a programmer needs to know about a code base while performing a change task and also on how they go about discovering that information. Based on a systematic analysis of the data from these user studies as well as an analysis of the support that current programming tools provide for these activities, this research makes four key contributions: (1) a catalog of 44 types of questions programmers ask, (2) a categorization of those questions into four categories based on the kind and scope of information needed to answer a question, (3) a description of important context for the process of answering questions, and (4) a description of support that is missing from current programming tools.

Software development remains mentally challenging despite the continual advancement of training, techniques, and tools. Because completely automating software development is currently impossible, it makes sense to seriously consider how tools can improve the mental activities of developers apart from automating them away. Such mental assistance can be called “cognitive support”. Understanding and developing cognitive support in software engineering tools is an important research issue but, unfortunately, at the moment our theoretical foundations for it are inadequately developed. Furthermore, much of the relevant research has occurred outside of the software engineering community, and is therefore not easily available to the researchers who typically develop software engineering tools. Tool evaluation, comparison, and development are consequently impaired. The present work introduces a theoretical framework intended to seed further systematic study of cognitive support in the field of software engineering tools. This theoretical framework, called RODS, imports ideas and methods from a field of cognitive science called “distributed cognition”. The crucial concept in RODS is that cognitive support can be understood and explained in terms of the computational advantages that are conferred when cognition is redistributed between software developer and their tools and environment. The name RODS, in fact, comes from the