A reachability question is a search across feasible paths through a program for target statements matching search criteria. In three separate studies, we found that reachability questions are common and often time consuming to answer. In the first study, we observed 13 developers in the lab and found that half of the bugs developers inserted were associated with reachability questions. In the second study, 460 professional software developers reported asking questions that may be answered using reachability questions more than 9 times a day, and 82 % rated one or more as at least somewhat hard to answer. In the third study, we observed 17 developers in the field and found that 9 of the 10 longest activities were associated with reachability questions. These findings suggest that answering reachability questions is an important source of difficulty understanding large, complex codebases.

Programmers seek to answer questions as they investigate the functioning of a software system, such as “which execution path is being taken in this case?” Programmers attempt to answer these questions, which we call conceptual queries, using a variety of tools. Each type of tool typically highlights one kind of information about the system, such as static structural information or control-flow information. Unfortunately for the programmer, the tools seldom directly answer the programmer’s conceptual queries. Instead, the programmer must piece together results from different tools to determine an answer to the initial query. At best, this process is time consuming and at worst, this process can lead to data overload and disorientation. In this paper, we present a model that supports the integration of different sources of information about a program. This model enables the results of concrete queries in separate tools to be brought together to directly answer many of a programmer’s conceptual queries. In addition to presenting this model, we present a tool that implements the model, demonstrate the range of conceptual queries supported by this tool, and present the results of use of the conceptual queries in a small field study.

Recent tools have been designed to help developers understand the potential runtime structure of objects in a system at compile time. Such tools let developers interactively explore diagrams of object structure. But do developers ask questions about object structure? If so, when? We conducted a small pilot study of developers working on coding tasks designed to require thinking about relationships between objects. Developers did indeed ask a number of questions about various types of relationships such as containment, ownership, object identities and aliasing. Finally, some of our results

To build new tools and programming languages that make it easier for professional software developers to create, debug, and understand code, it is helpful to better understand the questions that developers ask during coding activities. We surveyed professional software developers and asked them to list hard-to-answer questions that they had recently asked about code. 179 respondents reported 371 questions. We then clustered these questions into 21 categories and 94 distinct questions. The most frequently reported categories dealt with intent and rationale – what does this code do, what is it intended to do, and why was it done this way? Many questions described very specific situations – e.g., what does the code do when an error occurs, how to refactor without breaking callers, or the implications of a specific change on security. These questions revealed opportunities for both existing research tools to help developers and for developing new languages and tools that make answering these questions easier.

Many software visualization (SoftVis) tools are continuously being developed by both researchers as well as software development companies. In order to determine if the developed tools are effective in helping their target users, it is desirable that they are exposed to a proper evaluation. Despite this, there is still lack of a general guideline on how these evaluations should be carried out and many of the tool developers perform very limited or no evaluation of their tools. Each person that carries out one evaluation, however, has experiences which, if shared, can guide future evaluators. This paper presents the lessons learned from evaluating over 20 SoftVis tools with over 90 users in five different studies spread on a period of over two years. The lessons covered include the selection of the tools, tasks, as well as evaluation participants. Other discussed points are related to the duration of the evaluation experiment, its location, the procedure followed when carrying out the experiment, as well as motivation of the participants. Finally, an analysis of the lessons learned is shown with the hope that these lessons will be of some assistance to future SoftVis tool evaluators. 1

Requirements traceability is a challenge for modern software projects where task dependencies and technical expertise are spread across system developers, abstract model representations such as use cases, and a myriad of code artefacts. This paper presents an approach that monitors the navigation trails left by developers when building code artefacts to realise project use cases. These trails are analysed to generate a relevance ranking of entities that constitute a traceability link between uses cases and code artefacts and the developers responsible for them. Investigation in a software development scenario shows that a range of use case traceability questions can be answered through visualisations which present ordered relevance lists of the entities associated with use cases and by the use of trace graphs where the size of nodes show the importance, or ’information centrality’, of system entities. 1.

Source code search is an important activity for programmers working on a change task to a software system. As part of a larger project to improve tool support for finding information in source code, we conducted a formative study in which programmers were asked to perform corrective tasks to a system they were initially unfamiliar with. Our analysis focused specifically on how programmers decide what to search for, and how they decide which results are relevant to their task. Based on our analysis, we present five observations about our participant’s approach to finding information and some of the challenges they faced. We also discuss the implications these observations have for the design of source code search tools. 1.

Observations of developers indicate that developers try to answer a variety of questions by searching across control flow paths through a program for statements matching search criteria. We believe that tools that better support this activity can help developers answer these questions more easily, quickly, and accurately.

API users experience significant difficulties when learning how to use APIs, but little is known about the strategies used to overcome these difficulties, the motivation for each strategy, or the trade-offs between the strategies. To better understand the information seeking strategies of API users, we conducted a study in which 20 participants were asked to complete programming tasks using unfamiliar APIs, with the documentation of the APIs and the Web as learning resources. We observed that participants used one of three different strategies when seeking for information on how to use APIs: some were more inclined to using the Web, others preferred the documentation of the APIs, and others combined both the Web and the documentation. We present the characteristics, motivation, and trade-offs between these strategies, and suggests new ideas for documentation and tools to facilitate the information-seeking process of API learners.

Software developers understanding and exploring code spend much of their time asking questions and searching for answers. Yet little has been known about the questions devel-‐ opers ask, the strategies used to answer them, and the challenges developers face. Through interviews, surveys, and observations, a series of 7 studies were conducted that begin to address this gap, contributing a better understanding of developers ’ tools, practices, prob-‐ lems, questions, and strategies, and a model of how developers reconstruct design decisions from code. A design process is described for using studies of developers ’ work to design more useful tools for developers. These studies reveal that reachability questions are a central part of understanding and ex-‐ ploring code. A reachability question is a search along paths through code. Developers ask reachability questions when reasoning about causality, ordering, type membership, repeti-‐ tion, and choice. For example, to debug a deadlock, a developer searched downstream for calls acquiring resources to reconstruct how and why resources were acquired. Existing tools make these questions challenging to answer by forcing developers to guess which