Theories in “soft” areas of psychology lack the cumulative character of scientific knowledge. They tend neither to be refuted nor corroborated, but instead merely fade away as people lose interest. Even though intrinsic subject matter difficulties (20 listed) contribute to this, the excessive reliance on significance testing is partly responsible, being a poor way of doing science. Karl Popper’s approach, with modifications, would be prophylactic. Since the null hypothesis is quasi-always false, tables summarizing research in terms of patterns of “significant differences” are little more than complex, causally uninterpretable outcomes of statistical power functions. Multiple paths to estimating numerical point values (“consistency tests”) are better, even if approximate with rough tolerances; and lacking this, ranges, orderings, second-order differences, curve peaks and valleys, and function forms should be used. Such methods are usual in developed sciences that seldom report statistical significance. Consistency tests of a conjectural taxometric model yielded 94 % success with zero false negatives.

Abstract not found

In social science, everything is somewhat correlated with everything (“crud factor”), so whether H0 is refuted depends solely on statistical power. In psychology, the directional counternull of interest, H*, is not equivalent to the substantive theory T, there being many plausible alternative explanations of a mere directional trend (weak use of significance tests). Testing against a predicted point value (the strong use of significant tests) can discorroborate T by refuting H*. If used thus to abandon T forthwith, it is too strong, not allowing for theoretical verisimilitude as distinguished from truth. Defense and amendment of an apparently falsified T are appropriate strategies only when T has accumulated a good track record (“money in the bank”) by making successful or near-miss predictions of low prior probability (Salmon’s “damn strange coincidences”). Two rough indexes are proposed for numerifying the track record, by considering jointly how intolerant (risky) and how close (accurate) are its predictions. For almost three quarters of a century, the received doctrine about appraising psychological theories has been to perform a statistical significance test. In the “soft ” areas (clinical, counseling, developmental, personality, and social psychology),

accuracy issues more directly. Moreover, this research attracts a great deal of attention because of what many take to be its dismal implications for the accuracy of human social reasoning. These implications are illusory, however, because an error is not the same thing as a "mistake. " An error is a judgment of an experimental stimulus that departs from a model of the judgment process. If this model is normative, then the error can be said to represent an incorrect judgment. A mistake, by contrast, is an incorrect judgment of a real-world stimulus and therefore more difficult to determine. Although errors can be highly informative about the process of judgment in general, they are not necessarily relevant to the content or accuracy of particular judgments, because errors in a laboratory may not be mistakes with respect to a broader, more realistic frame of reference and the processes that produce such errors might lead to correct decisions and adaptive outcomes in real life. Several examples are described in this article. Accuracy issues cannot be addressed by research that concentrates on demonstrating error in relation to artificial stimuli, but only by research that uses external, realistic criteria for accuracy. These criteria might include the degree to which judgments agree with each other and yield valid predictions of behavior. The accuracy of human social judgment is a topic of obvious

When time is limited, researchers may be faced with the choice of using an extremely brief measure of the Big-Five personality dimensions or using no measure at all. To meet the need for a very brief measure, 5 and 10-item inventories were developed and evaluated. Although somewhat inferior to standard multi-item instruments, the instruments reached adequate levels in terms of: (a) convergence with widely used Big-Five measures in self, observer, and peer reports, (b) test–retest reliability, (c) patterns of predicted external correlates, and (d) convergence between self and observer ratings. On the basis of these tests, a 10-item measure of the Big-Five dimensions is offered for situations where very short measures are needed, personality is not the primary topic of interest, or researchers can tolerate the somewhat diminished psychometric properties associated with very brief measures.

In recent years, several organizations have implemented non-mandatory information and communication systems that escape the conventional behavioral logic of understanding acceptance and usage from a normative perspective of compliance with the beliefs of others. Because voluntary systems require users' volitional behavior, researchers have traced recent implementation failures to a lack of user commitment. However, gaps in our understanding of volitional usage behavior and user commitment have made it difficult to advance theory, research, and practice on this issue. To validate a proposed research model, cross-sectional, between-subjects, and within-subjects field data were collected from 714 users at the time of initial adoption and after six months of extended use. The model explained between 44.1% and 58.5% of the variance in adoption and usage behavior based upon direct effects of user commitment. Findings suggest that user commitment plays a critical role in the volitional acceptance and usage of such systems. Affective commitment, i.e., internalization and identification based upon personal norms, exhibits a sustained positive influence on usage behavior. In contrast, continuance commitment, i.e., compliance based upon social norms, shows a sustained negative influence from initial adoption to extended use. Theory development based upon Kelman's social influence framework offers new empirical insights about system users' commitment and how it affects volitional usage behavior.

In this study a psychometric instrument specifically designed to measure human-computer trust (HCT) was developed and tested. A rigorous method similar to that described by Moore and Benbasat (1991) was adopted. It was found that both cognitive and affective components of trust could be measured and that, in this study, the affective components were the strongest indicators of trust. The reliability of the instrument, measured as Cronbach's alpha, was 0.94. This instrument is the first of its kind to be specifically designed to measure HCT and shown empirically to be valid and reliable.

This article advances a simple conception of test validity: A test is valid for measuring an attribute if (a) the attribute exists and (b) variations in the attribute causally produce variation in the measurement outcomes. This conception is shown to diverge from current validity theory in several respects. In particular, the emphasis in the proposed conception is on ontology, reference, and causality, whereas current validity theory focuses on epistemology, meaning, and correlation. It is argued that the proposed conception is not only simpler but also theoretically superior to the position taken in the existing literature. Further, it has clear theoretical and practical implications for validation research. Most important, validation research must not be directed at the relation between the measured attribute and other attributes but at the processes that convey the effect of the measured attribute on the test scores. We start this article with a request to the reader. Please take a slip of paper and write down your definition of the term construct validity. Now, take the classic article of Cronbach and Meehl (1955), who invented the concept, and a more recent authoritative article on validity, for instance that of Messick (1989), and check whether you recognize your definition in these works. You are

This paper analyzes the theoretical, pragmatic, and substantive factors that have hampered the integration between psychology and psychometrics. Theoretical factors include the operationalist mode of thinking which is common throughout psychology, the dominance of classical test theory, and the use of “construct validity ” as a catch-all category for a range of challenging psychometric problems. Pragmatic factors include the lack of interest in mathematically precise thinking in psychology, inadequate representation of psychometric modeling in major statistics programs, and insufficient mathematical training in the psychological curriculum. Substantive factors relate to the absence of psychological theories that are sufficiently strong to motivate the structure of psychometric models. Following the identification of these problems, a number of promising recent developments are discussed, and suggestions are made to further the integration of psychology and psychometrics.

Exploratory Studies on the Sense of Presence as a Function of Visual and Auditory Display Parameters in Virtual Environments by Claudia Mary Hendrix Chairperson of the Supervisory Committee: Professor Woodrow Barfield Department of Industrial Engineering The focus of this research was to develop subjective measures of presence in order to relate the amount of presence experienced within a virtual environment to visual and auditory display parameters. Furthermore, the research examined the relationship between presence and performance in spatial judgments. To accomplish this goal three experiments were performed; the first two experiments investigated virtual presence as a function of auditory and visual display parameters used to design virtual environments while the third study investigated presence as a function of performance measures and visual display parameters. The first experiment investigated the sense of presence within virtual environments as a function of visual display parameters while the second investigated the effect of auditory display parameters. The variables for the first experiment included the presence or absence of head tracking, the presence or absence of stereoscopic cues, and the geometric field of view (GFOV) used to design the visual display. The variables for the second experiment included the presence or absence of spatialized sound to a stereoscopic virtual environment display and the addition of spatialized versus non-spatialized sound to a stereoscopic virtual environment display. Subjects were required to navigate a virtual environment and to complete a questionnaire designed to ascertain the level of presence experienced by the participant within the virtual world. The results indicated that the reported level of presence was significa...