As programming skills increase in demand and utility, the learnability of end-user programming systems is of utmost importance. However, research on learning barriers in programming systems has primarily focused on languages, overlooking potential barriers in the environment and accompanying libraries. To address this, a study of beginning programmers learning Visual Basic.NET was performed. This identified six types of barriers: design, selection, coordination, use, understanding, and information. These barriers inspire a new metaphor of computation, which provides a more learner-centric view of programming system design.

A study by a ITiCSE 2001 working group ("the McCracken Group") established that many students do not know how to program at the conclusion of their introductory courses. A popular explanation for this incapacity is that the students lack the ability to problem-solve. That is, they lack the ability to take a problem description, decompose it into sub-problems and implement them, then assemble the pieces into a complete solution. An alternative explanation is that many students have a fragile grasp of both basic programming principles and the ability to systematically carry out routine programming tasks, such as tracing (or “desk checking”) through code. This ITiCSE 2004 working group studied the alternative explanation, by testing

An essential aspect of programmers' work is the correctness of their code. This makes current HCI techniques ichnikTz to analyze and desiW the programmiz systems that programmers use everyday,siry thesetechnixb% focus more on problemswib learnabifi%x andefficiqxk of use, and less on error-proneness. We propose a frameworkand methodology that focusesspeci-q-%b on errors bysupportiz thedescrizkTz andikb-zW-kTzzW of the causes of software errorsi terms ofchaib ofcogniqfik breakdowns. The frameworki based on both old and newstudiW ofprogrammiqfi as well as general research on themechanikT of human error. Our experi%kT- usie the frameworkand methodology to study theAlib programmiT system havediek-W% iek-W% thedesib of several newprogrammitools and idkfi-WbWkT Thi ii-Wb theWhyliO debuggib iuggibkT whi we have shown to reduce debuggifi tiu by a factor of 8 and help programmers get 40% further throughthei tasks. WediOEfi% the framework's and methodology'sithodology' forprogrammisystem desimk softwareengirek%EbO and the psychology of programming.

Type checking is considered an important mechanism for detecting programming errors, especially interface errors. This report describes an experiment to assess the defect-detection capabilities of static, inter-module type checking. The experiment uses Ansi C and Kernighan&Ritchie (K&R) C. The relevant difference is that the Ansi C compiler checks module interfaces (i.e., the parameter lists of calls to external functions), whereas K&R C does not. The experiment employs a counterbalanced design in which each of the 40 subjects, most of them CS Ph.D. students, writes two non-trivial programs that interface with a complex library (Motif). Each subject writes one program in Ansi C and one in K&R C. The input to each compiler run is saved and manually analyzed for defects. Results indicate that delivered Ansi C programs contain significantly fewer interface defects than delivered K&R C programs. Furthermore, after subjects have gained some familiarity with the interface they are using, An...

In previous work we have developed a system that automatically checks for unit errors in spreadsheets. In this paper we describe our experiences using the system in a workshop on spreadsheet safety aimed at high school teachers and students. We present the results from a think-aloud study we conducted with five high school teachers and one high school student as the subjects. The study is the first ever to investigate the usability of a type system in spreadsheets. We discovered that end users can effectively use the system to debug a variety of errors in their spreadsheets. This result is encouraging given that type systems are difficult even for programmers. The subjects had difficulty debugging “non-local” unit errors. Guided by the results of the study we devised new methods to improve the errorlocation inference. We also extended the system to generate change suggestions for cells with unit errors, which when applied, would correct unit errors. These extensions solved the problem that the study revealed in the original system.

This paper reports on an experiment to assess the error detection capabilities of static, inter-module type checking. Type checking is considered an important mechanism for detecting programming errors, especially interface errors. The experiment uses Kernighan&Ritchie C and ANSI C. The relevant difference is that the ANSI C compiler checks module interfaces (i.e., the parameter lists of calls to external functions), whereas K&R C does not. The experiment employs a counterbalanced design, in which each subject writes two non-trivial programs that interface with a complex library (Motif). Each subject writes one program in K&R C and one in ANSI C. The input to each compiler run is saved and manually analyzed for errors. Results indicate that delivered ANSI C programs contain significantly fewer interface errors than delivered K&R C programs. Furthermore, after subjects have gained some familiarity with the interface they are using, ANSI C programmers remove errors faster and are more pr...

Empirical studies of novice programming typically rely on code solutions or test responses as the basis of their analyses. While such data can provide insight into novice programming knowledge, they say little about the programming processes in which novices engage. For those interested in improving novice programming environments, a key research question arises: How can we collect and analyze data on novice programming that will enable us (a) to analyze and compare the programming processes promoted by alternative novice programming environments, and (b) ultimately to build better novice programming environments? To address this question, we have collected a large video corpus of novices as they construct code solutions in various versions of ALVIS Live! [14], a novice programming environment. Through detailed post-hoc analyses of our video corpus, we have developed a methodology for compiling the moment-by-moment evolution of novice code solutions. Based on an analysis of an ideal code solution’s key semantic components, our methodology enables one to document, on a second-by-second basis, (a) what part of a code solution a programmer is focusing on, and (b) where the semantic feedback provided by the programming environment is helping. Although it is time and labor intensive, our methodology provides researchers with a standard set of data and representations for comparing the programming processes promoted by alternative programming environments.

: A number of Program Visualisation (PV) systems have been developed to aid the debugging process by providing a detailed account of program execution. Despite their development, empirical claims for their efficacy are few. A review of related empirical evaluations identifies the lack of a principled and integrated approach as being a significant problem. This criticism combined with a description of SV within the context of the programming environment is used to develop a set of methodological criteria for future work. These requirements have motivated the implementation of the Prolog Program Visualization Laboratory (PPVL) which provides an integrated tracing environment comprising four Prolog tracers. An information-based methodology is then presented for evaluating Prolog tracers within the realistic programming environment provided by PPVL. Finally, a survey of PV design claims are used to focus future research. Keywords: Program Visualisation, Evaluation, Methodology, HCI. Ackn...

Programming may be more difficult than necessary because it requires solutions to be expressed in ways that are not familiar or natural for beginners. To identify what is natural, this article examines the ways that non-programmers express solutions to problems that were chosen to be representative of common programming tasks. The vocabulary and structure in these solutions is compared with the vocabulary and structure in modern programming languages, to identify the features and paradigms that seem to match these natural tendencies as well as those that do not. This information can be used by the designers of future programming languages to guide the selection and generation of language features. This design technique can result in languages that are easier to learn and use, because the languages will better match beginners' existing problem solving abilities.

Abstract approved: