Claims for the virtues of visual programming languages have generally been strong, simple-minded statements that visual programs are inherently better than textual ones. They have paid scant attention to previous empirical literature showing difficulties in comprehending visual programs. This paper reports comparisons between the comprehensibility of textual and visual programs, drawing on the methods developed by Green (1977) for comparing detailed comprehensibility of conditional structures. The visual language studied was LabView, a circuit-diagram-like language which can express conditionals either as `forwards' structures (condition implies action, with nesting) or as `backwards' structures (action is governed by conditions, with boolean operators in place of nesting). Green (1977) found that forwards structures gave relatively better access to `sequential' information, and Gilmore and Green (1984) showed `backwards' structures gave relatively better access to `circumstantial' inf...

Recently we have witnessed a welcomed increase in the amount of empirical evaluation of Software Engineering methods and concepts. It is hoped that this increase will lead to establishing Software Engineering as a well-defined subject with a sound scientifically proven underpinning rather than a topic based upon unsubstantiated theories and personal belief. For this to happen the empirical work must be of the highest standard. Unfortunately producing meaningful empirical evaluations is a highly hazardous activity, full of uncertainties and often unseen difficulties. Any researcher can overlook or neglect a seemingly innocuous factor, which in fact invalidates all of the work. More serious is that large sections of the communuity can overlook essential experimental design guidelines, which bring into question the validity of much of the work undertaken to date. In this paper, the authors address one such factor - Statistical Power Analysis. It is believed and will be demonstrated that a...

The role of replication in experimental computer science is discussed. Various kinds of internal and external replication are differentiated. Without the confirming power of external replications, results in experimental computer science should only be provisionally accepted. An extension to Basili et al's framework for experimentation in software engineering is proposed to more fully differentiate between the various kinds of replication and their powers of confirmation. Guidance is also given to the level of reported detail required to better enable others to perform a meaningful replication. It is concluded that external replications of experiments in Computer Science have a vital part to play in the realisation of generalizable results. Keywords : empirical, experimental, external, repeatable, replication, reproducable 1 Introduction Experimentation within Computer Science is not an exact process. Experimental design is difficult and the experimental process can be error prone. As...

The importance of empirical studies in software engineering cannot be understated. Care must be taken, however, that the experimental methods employed are appropriate and that they are executed with due scientific rigour. A critical commentary is presented of a paper by Sinha and Vessey [6] which attempted empirical evaluations of a number of propositions concerned with the notion of cognitive fit (i.e. the notion that problem solving can be facilitated by use of appropriate representations and tools.) There are concerns about the scale, nature, and representation of the experimental tasks. The entire experiment relies on the concept of a naturally recursive task. There are also concerns about the lack of information about the subjects and their use of the programming environments. Some criticisms are also presented of the analysis of the results and the discussion. It is concluded that the interpretations given in [6] carry no weight. Keywords : cognitive, criticisms, empirical, itera...