Abstract—Systems exhibiting relationshipsbetween mental states and material states, briefly mind-matter systems, offer epistemological and methodological problems exceeding those of systems involving mental states or material states alone. Some of these problems can be addressed by proceeding from standard first-order approaches to more sophisticated second-order approaches. These can illuminate questions of reference and validity, and their ramifications for the topic of reproducibility. For various situations in complex systems it is shown that second-order approaches need to be employed. Considering mindmatter systems as generalized complex systems provides some guidelines for analyzing the problem of reproducibility in such systems from a novel perspective.

Does psi exist? In a recent article, Dr. Bem conducted nine studies with over a thousand participants in an attempt to demonstrate that future events retroactively affect people’s responses. Here we discuss several limitations of Bem’s experiments on psi; in particular, we show that the data analysis was partly exploratory, and that one-sided p-values may overstate the statistical evidence against the null hypothesis. We reanalyze Bem’s data using a default Bayesian t-test and show that the evidence for psi is weak to nonexistent. We argue that in order to convince a skeptical audience of a controversial claim, one needs to conduct strictly confirmatory studies and analyze the results with statistical tests that are conservative rather than liberal. We conclude that Bem’s p-values do not indicate evidence in favor of precognition; instead, they indicate that experimental psychologists need to change the way they conduct their experiments and analyze their data. Keywords: Confirmatory Experiments, Bayesian Hypothesis Test, ESP. In a recent article for Journal of Personality and Social Psychology, Bem (in press) presented nine experiments that test for the presence of psi. 1 Specifically, the experiments were designed to assess the hypothesis that future events affect people’s thinking and people’s behavior in the past (henceforth precognition). As indicated by Bem, precognition—if it exists—is an anomalous phenomenon, because it conflicts with what we know to be true about the word (e.g., weather forecasting agencies do not employ clairvoyants, casino’s 1 The preprint that this article is based on was downloaded September 25th, 2010, from

conducted an evaluation of a 24-year, government-sponsored program to investigate ESP and its potential use within the Intelligence Community. The American Institutes for Research was contracted to conduct the review of both research and operations. Their 29 September 1995 final report was released to the public 28 November 1995. As a result of AIR's assessment, the CIA concluded that a statistically significant effect had been demonstrated in the laboratory, but that there was no case in which ESP had provided data that had ever been used to guide intelligence operations. This paper is a critical review of AIR's methodology and conclusions. It will be shown that there is compelling evidence that the CIA set the outcome with regard to intelligence usage before the evaluation had begun. This was accomplished by limiting the research and operations data sets to exclude positive findings, by purposefully not interviewing historically significant participants, by ignoring previous DOD extensive program reviews, and by using the questionable National Research Council's investigation of parapsychology as the starting point for their review. While there may have been political and administrative justification for the CIA not to accept the government's in-house program for the operational use of anomalous cognition, this appeared to drive the outcome of the evaluation. As a result, they have come to the wrong conclusion with regard to the use of anomalous cognition in intelligence operations and significantly underestimated the robustness of the basic phenomenon. Executive Summary

Studies of anomalous correlations between mind and matter usually focus on participating subjects and isolated target systems. We report on a decade-long experiment which finds that anomalous mind-matter correlations may be a pervasive aspect of reality. The Global Consciousness Project (GCP) measures the output deviation of a global network of physical random number generators (RNG) at the time of major world events. The project hypothesizes that the coherent attention or emotional response of large populations induced by the events will correspond to characteristic deviations of the network output. We describe the motivation and scope of the experiment and the analytical procedures employed to test the hypothesis, and present the results of 236 events accumulated over the first nine years of operation. The cumulative significance across all events favors the hypothesis by more than 4.5 standard deviations. Beyond a test of the basic hypothesis, secondary analyses show that the result is driven by correlations in the RNG network across global distances. 1 GCP Event Experiment, Bancel and Nelson, 3/14/2008, preprint (JSE), Do not distribute 1.

Statistical methods are designed to detect and measure relationships and effects in situations where results cannot be identically replicated because of natural variability in the measurements of interest. They are generally used as an intermediate step between anecdotal evidence and the determination of causal explanations. Many anomalous phenomena, such as remote viewing or the possible effect of prayer on healing, are amenable to rigorous study. Statistical methods play a major role in making appropriate conclusions from those studies. This paper examines the role statistics can play in summarizing and drawing conclusions from individual and collective studies. Two examples of using meta-analysis to assess evidence are presented and compared. One is a conventional example relating the use of antiplatelets to reduced vascular disease, and the other is an example from mind-matter research, illustrating results of ganzfeld and remote viewing experiments.

Mind-matter interactions observed in laboratory experiments typically manifest as minute statistical fluctuations from chance expectation. Meta-analyses suggest that these small fluctuations reflect genuine, direct interactions between mind and matter, but lack of predictability of the effect has made systematic study of the phenomenon difficult. Two general factors that may contribute to erratic laboratory outcomes are (a) unavoidable environmental fluctuations and (b) a physical principle that tends to counterbalance mind-matter effects in time and space. Environment is used in the holistic sense, including cosmic, global, local, and personal variables. The physical principal is envisioned as a tendency for perturbations introduced into a system to be statistically balanced by opposing perturbations so as to maintain an overall condition of equilibrium. A longitudinal experiment with the experimenter as subject was designed to explore the possible influences of these two factors in mind-matter interaction. The results found strong indicators of environmental modulation, including a successful demonstration that a neural network could learn to predict mind-matter interaction performance based upon eight environmental variables. Evidence for a time- and space-like equilibrium principle was also observed in the data.

Advocates and promoters of specific education methods are heard to say Òthe research shows that different teaching pedagogy really matters.Ó Education specialist Ramsden (1998) asserts: ÒThe picture of what encourages students to learn effectively at university is now almost complete.Ó(p. 355) Anecdotal evidence and arguments based on theory are often provided to support such claims, but quantitative studies of the effects of one teaching method versus another are either not cited or are few in number. DeNeve and Heppner (1997), for example, found only 12 of the 175 studies identified in a 1992-1995 search for Òactive learningÓ in the Educational Resources Information Center (ERIC) data base made comparisons of active learning techniques with another teaching method. An ERIC search for ÒClassroom Assessment TechniquesÓ (CATs) undertaken for me by Jillian Kinzicat yielded a similar outcome. My own (March 2000) request to CATs specialist Tom Angelo for direction to quantitative studies supporting the effectiveness of CATs yielded some good leads, but in the end there were few quantitative studies employing inferential statistics. Even when references to quantitative studies are provided, they typically