Although detractors of functional programming sometimes claim that functional programming is too difficult or counterintuitive for most programmers to understand and use, evidence to the contrary can be found by looking at the popularity of spreadsheets. The spreadsheet paradigm, a first-order subset of the functional programming paradigm, has found wide acceptance among both programmers and end users. Still, there are many limitations with most spreadsheet systems.

This article presents a testing methodology that adapts data flow adequacy criteria and coverage monitoring to the task of testing spreadsheets. To accommodate the evaluation model used with spreadsheets, and the interactive process by which they are created, our methodology is incremental. To accommodate the users of spreadsheet languages, we provide an interface to our methodology that does not require an understanding of testing theory. We have implemented our testing methodology in the context of the Forms/3 visual spreadsheet language. We report on the methodology, its time and space costs, and the mapping from the testing strategy to the user interface. In an empirical study, we found that test suites created according to our methodology detected, on average, 81% of the faults in a set of faulty spreadsheets, significantly outperforming randomly generated test suites

Information has become interactive. Information visualization is the design and creation of interactive graphic depictions of information by combining principles in the disciplines of graphic design, cognitive science, and interactive computer graphics. As the volume and complexity of the data increases, users require more powerful visualization tools that allow them to more effectively explore large abstract datasets. This

Form-based visual programming languages, which include commercial spreadsheets and various research systems, have had a substantial impact on end-user computing. Research shows, however, that form-based visual programs often contain faults. We would like to provide at least some of the benefits of formal testing methodologies to the creators of these programs. This paper presents a testing methodology for form-based visual programs. To accommodate the evaluation model used with these programs, and the interactive process by which they are created, our methodology is validationdriven and incremental. To accommodate the users of these languages, we provide an interface to the methodology that does not require an understanding of testing theory. We discuss our implementation of this methodology and empirical results achieved in its use.

Is it possible to achieve some of the benefits of formal testing within the informal programming conventions of the spreadsheet paradigm? We have been working on an approach that attempts to do so via the development of a testing methodology for this paradigm. Our "What You See Is What You Test" (WYSIWYT) methodology supplements the convention by which spreadsheets provide automatic immediate visual feedback about values by providing automatic immediate visual feedback about "testedness". In previous work we described this methodology; in this paper, we present empirical data about the methodology's effectiveness. Our results show that the use of the methodology was associated with significant improvement in testing effectiveness and efficiency, even with no training on the theory of testing or test adequacy that the model implements. These results may be due at least in part to the fact that use of the methodology was associated with a significant reduction in overconfidence. Keywor...

In the past, attempts to extend the spreadsheet paradigm to support graphical objects, such as colored circles or user-defined graphical types, have led to approaches featuring either a direct way of creating objects graphically or strong compatibility with the spreadsheet paradigm, but not both. This inability to conveniently go beyond numbers and strings without straying outside the spreadsheet paradigm has been a limiting factor in the applicability of spreadsheet languages. In this paper we present graphical definitions, an approach that removes this limitation, allowing both simple and complex graphical objects to be programmed directly using direct manipulation and gestures, in a manner that fits seamlessly within the spreadsheet paradigm. We also describe an empirical study, in which subjects programmed such objects faster and with fewer errors using this approach than when using a traditional approach to formula specification. Because the approach is expressive enough to be used with both built-in and user-defined types, it allows the directness of demonstrational and spreadsheet techniques to be used in programming a wider range of applications than has been possible before.

Despite their ability to help with program correctness, assertions have been notoriously unpopular---even with professional programmers. End-user programmers seem even less likely to appreciate the value of assertions; yet end-user programs suffer from serious correctness problems that assertions could help detect. This leads to the following question: can end users be enticed to enter assertions? To investigate this question, we have devised a curiosity-centered approach to eliciting assertions from end users, built on a surprise-explain-reward strategy. Our follow-up work with end-user participants shows that the approach is effective in encouraging end users to enter assertions that help them find errors.

There has been little research on end-user program development beyond the activity of programming. Devising ways to address additional activities related to end-user program development may be critical, however, because research shows that a large proportion of the programs written by end users contain faults. Toward this end, we have been working on ways to provide formal "software engineering" methodologies to end-user programmers. This paper describes an approach we have developed for supporting assertions in end-user software, focusing on the spreadsheet paradigm. We also report the results of a controlled experiment, with 59 end-user subjects, to investigate the usefulness of this approach. Our results show that the end users were able to use the assertions to reason about their spreadsheets, and that doing so was tied to both greater correctness and greater efficiency.

People find it difficult to create and maintain abstractions. We often deal with abstract tasks by using notations that make the structure of the abstraction visible. PBE systems sometimes make it more difficult to create abstractions. The user has to second-guess the results of the inference algorithm, and sometimes cannot see any visual representation of the inferred result, let alone manipulate it easily. SWYN (See What You Need) addresses these issues in the context of constructing regular expressions from examples. It provides a visual representation that has been evaluated in empirical user testing, and an induction interface that always allows the user to see and modify the effects of the supplied examples. The results demonstrate the potential advantages of more strictly applying cognitive dimensions analysis and direct manipulation principles when designing systems for programming by example.

Although there has been considerable research into ways to design visual programming environments to improve the processes of creating new programs and of understanding existing ones, little attention has been given to helping users of these environments test their programs. This feature would be particularly important for systems aimed at end users, since testing is the primary device they use to determine whether their programs are correct. To help address this need, we introduce two visual approaches to testing large grids in spreadsheet systems. This work scales up a visual testing methodology we previously developed for individual cells. The approaches are tightly integrated into Forms/3, a visual spreadsheet language, and communication with the user happens solely through the use of checkbox devices and coloring mechanisms. The intent of this work is to bring to end users at least some of the benefits of formalized notions of testing, without requiring knowledge of testing beyond a naive level. 1.