After 4 decades of severe criticism, the ritual of null hypothesis significance testing—mechanical dichotomous decisions around a sacred.05 criterion—still persists. This article reviews the problems with this practice, including its near-universal misinterpretation ofp as the probability that Ho is false, the misinterpretation that its complement is the probability of successful replication, and the mistaken assumption that if one rejects Ho one thereby affirms the theory that led to the test. Exploratory data analysis and the use of graphic methods, a steady improvement in and a movement toward standardization in measurement, an emphasis on estimating effect sizes using confidence intervals, and the informed use of available statistical methods is suggested. For generalization, psychologists must finally rely, as has been done in all the older sciences,

This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) and Ihe 18 other selection procedures. Overall, the 3 combinations with the highest multivariate validity and utility for job performance were GMA plus a work sample test (mean validity of.63), GMA plus an integrity test (mean validity of.65), and GMA plus a structured interview (mean validity of.63). A further advantage of the latter 2 combinations is that they can be used for both entry level selection and selection of experienced employees. The practical utility implications of these summary findings are substantial. The implications of these research findings for the development of theories of job performance are discussed. From the point of view of practical value, the most important property of a personnel assessment method is predictive validity: the ability to predict future job performance, job-related learning (such as amount of learning in training and development programs), and other criteria. The predictive validity coefficient is

Abstract not found

This article is concerned with the implications of the surprisingly different experimental practices in economics and in areas of psychology relevant to both economists and psychologists, such as behavioral decision making. We consider four features of experimentation in economics, namely, script enactment, repeated trials, performance-based monetary payments, and the proscription against deception, and compare them to experimental practices in psychology, primarily in the area of behavioral decision making. Whereas economists bring a precisely defined ìscriptî to experiments for

Abstract—Although projective techniques continue to be widely used in clinical and forensic settings, their scientific status remains highly controversial. In this monograph, we review the current state of the literature concerning the psychometric properties (norms, reliability, validity, incremental validity, treatment utility) of three major projective instruments: Rorschach Inkblot Test, Thematic Apperception Test (TAT), and human figure drawings. We conclude that there is empirical support for the validity of a small number of indexes derived from the Rorschach and TAT. However, the substantial majority of Rorschach and TAT indexes are not empirically supported. The validity evidence for human figure drawings is even more limited. With a few exceptions, projective indexes have not consistently demonstrated incremental validity above and beyond other psychometric data. In addition, we summarize

In first explicating the notion of quantitative literature review for the social sciences, Glass (1976) argued that knowledge is not built from any individual study, but from the integration of findings from many studies. Individual studies do not so much yield knowledge as evidence with which knowledge can be built. Knowledge is socially constructed. To overemphasize a single study’s findings or integrate research only impressionistically leaves researchers knowing less than the evidence offers, insufficiently exploiting the wealth of data scattered in separate studies. Quantitative research integration, or meta-analysis, has a history in both the physical and social sciences that precedes Glass ’ formulation (Bangert-Drowns 1986; Hedges 1987). Most generally, meta-analysis is a perspective rather than a method, a recognition that research findings can be interpreted probabilistically in the context of collections of studies. The meta-analytic perspective is consistent with,

This monograph is based on the papers from a technical review on

This document was prepared as a resource for researchers attending the meeting "Assessing the Impact of Childhood Interventions on Subsequent Drug Abuse " organized by the National Institute on Drug Abuse with co-sponsorship by the National Institute of Mental Health. Little is known about how mental health treatments for childhood psychopathologies influence the subsequent risk of drug abuse.

Abstract. Rituals shape many aspects of our lives, and they are no less common in scientific research than elsewhere. One that figures prominently in hypothesis testing is the null ritual, the pitting of hypotheses against chance. Although known to be problematic, this practice is still widely used. One way to resist the lure of the null ritual is to increase the precision of theories by casting them as formal models. These can be tested against each other, instead of against chance, which in turn enables a researcher to decide between competing theories based on quantitative measures. This article gives an overview of the advantages of modeling, describes research that is based on it, outlines the difficulties associated with model testing, and summarizes some of the solutions for dealing with these difficulties. Pointers to resources for teaching modeling in university classes are provided.

Statistics education in psychology often falls disappointingly short of its goals. The increasing use of qualitative approaches in statistics education research has extended and enriched our understanding of statistical cognition processes, and thus facilitated improvements in statistical education and practices. Yet conceptual analysis, a fundamental part of the scientific method and arguably the primary qualitative method insofar as it is logically prior and equally applicable to all other empirical research methods—quantitative, qualitative, and mixed—has been largely overlooked. In this paper we present the case for this approach, and then report results from a conceptual analysis of statistics education in psychology. The results highlight a number of major problems that have received little attention in standard statistics education research.