ABSTRACT: The Likelihood Principle has been defended on Bayesian grounds, on the grounds that it coincides with and systematizes intuitive judgments about example problems, and by appeal to the fact that it generalizes what is true when hypotheses have deductive consequences about observations. Here we divide the Principle into two parts-- one qualitative, the other quantitative-- and evaluate each in the light of the Akaike information criterion. Both turn out to be correct in a special case (when the competing hypotheses have the same number of adjustable parameters), but not otherwise.

In the 1960s, Kuhn maintained that there is no standard higher of rationality than the assent of the relevant community. Realists have seek to evaluate the rationality of science relative to a highest standard possible—namely the truth, or approximate truth, of our best theories. Given that the realist view of rationality is controversial, it seems that a more secure reply to Kuhn should be based on a less controversial objective of science—namely, the goal of predictive accuracy. Not only does this yield a more secure reply to Kuhn, but it also provides the foundation on which any realist arguments should be based. In order to make this case, it is necessary to introduce a three-way distinction between theories, models, and predictive hypotheses, and then ask some hard questions about how the methods of science can actually achieve their goals. As one example of the success of such a program, I explain how the truth of models can sometimes lower their predictive accuracy. As a second example, I describe how one can define progress across paradigms in terms of predictive accuracy. These are examples of hard problems in the philosophy of science, which fall outside the scope of social psychology.

Akaike's framework for thinking about model selection in terms of the goal of predictive accuracy and his criterion for model selection have important philosophical implications. Scientists sometimes test models whose truth values they already know, and then sometimes choose models that they know full well are false. Instrumentalism helps explain this pervasive feature of scientific practice, and Akaike's framework provides instrumentalism with the epistemology it needs. Akaike's criterion for model selection also throws light on the role of parsimony considerations in hypothesis evaluation. I explain the basic ideas behind Akaike's framework and criterion; several biological examples, including the use of maximum likelihood methods in phylogenetic inference, are considered. Philosophers of science usually agree that the point of testing theories -- indeed, the point of doing science -- is to try to determine which theories are true. Of course, we all recognize that scientists never have access to all possible theories on a given subject; they are limited by the theories they have at hand. But given a set of competing theories, the point of theory assessment is to ascertain which of these competitors is one's best guess as to what the truth is. Bayesians tend to see things this way, so do scientists who use orthodox Neyman-Pearson methods, and likelihoodists tend to fall into this pattern as well. To be sure, there are deep differences among these outlooks. Bayesians assess which hypotheses are most probable, frequentists evaluate which hypotheses should be rejected, and likelihoodists say which hypotheses are best supported. But these assessments typically invoke the concept of truth; the question is which hypotheses are most probably true, or should be rejected as ...

Abstract not found

Balancing comfort: Occupants' control of window blinds in private offices by Vorapat Inkarojrit Doctor of Philosophy in Architecture University of California, Berkeley Professor Charles C. Benton, Chair The goal of this study was to develop predictive models of window blind control that could be used as a function in energy simulation programs and provide the basis for the development of future automated shading systems. Toward this goal, a two-part study, consisting of a window blind usage survey and a field study, was conducted in Berkeley, California, USA, during a period spanning from the vernal equinox to window solstice. A total of one hundred and thirteen office building occupants participated in the survey. Twenty-five occupants participated in the field study, in which measurements of physical environmental conditions were cross-linked to the participants' assessment of visual and thermal comfort sensations. Results from the survey showed that the primary window blind closing reason was to reduce glare from sunlight and bright windows. For the field study, a total of thirteen predictive window blind control logistic models were derived using the Generalized Estimating Equations (GEE) technique. TABLE OF CONTENTS TABLE OF CONTENTS........................................................................................... i LIST OF FIGURES................................................................................................... v LIST OF TABLES..................................................................................................... xi ACKNOWLEDGEMENTS.......................................................................................xiii CHAPTER 1:

Provenance of correlations in psychological data There are few truisms in the field of psychology, but one of them is surely that measurement error is found in all experiments. Data are inevitably produced that do not perfectly reflect the logic imposed by the experimental design. To the extent that a psychological experiment succeeds in measuring something or in making some sort of distinction, the data will partially reflect the design, and this leads to a way of thinking about data that is found throughout all the experimental sciences: data � signal � noise. This innocent equation almost always contains an implicit but critical assumption: that the noise may be regarded as independent samples from some distribution— typically taken to be the Gaussian distribution. In this way, the residual error is conceived of as a featureless background of white noise in which the interesting part, the treatment means, are more or less buried. Often this conception of data is justified. Whenever there is random assignment to cells and each participant contributes a single datum, errors may be expected to be uncorrelated. However, in all of sensory psychophysics and most of cognitive psychology, single individuals respond to entire blocks of trials in a given experimental session. Here, the residual error will develop correlations by virtue of the circumstance that the response history was laid down by a nervous system that has memory. In many situations, these correlations are little more than a Preparation of this article was supported by NIMH Grants R01-

Wayne Myrvold (2003) has captured an important feature of unified theories, and he has done so in Bayesian terms. What is not clear is whether the virtue of such unification is most clearly understood in terms of Bayesian confirmation. I argue that the virtue of such unification is better understood in terms of other truth-related virtues such as predictive accuracy. † Send requests for reprints to: email: mforster @ wisc.edu; homepage:

Abstract: Bayesians often reject the Akaike Information Criterion (AIC) because it introduces ideas that do not fit into their philosophy of statistical inference. Here we show that a difference in the AIC scores that two models receive is evidence that they differ in their degrees of predictive accuracy, where evidence is understood in terms of the Law of Likelihood. Since the Law of Likelihood is a central Bayesian principle,

A model selection method inspired by the idea of universal models is proposed for choosing between spatially explicit multi-agent decision models. Since this kind of models are often used as support in real-world decision making, and usually there are not much data available to validate the models, it is important that the selection of the best model is based on sound criteria, not merely on model's face value. In this research the best model is de ned as one with an adequate fit to the current data and good generalizability to the future data. The criterion's ability to choose the best model is achieved by properly combining measures for goodness of fit and complexity in order to avoid over- tting, i.e., the models ability to fit noise in the data, induced by its super uous complexity. The proposed model selection criterion is applied and its performance is analyzed in the context of simple spatial agent-based decision models, which are made to generate arti cial data. The preliminary results indicate that the criterion adequately balances the two measures by preferring the simpler model when there are not much data available, even when the more complex model generated the data.

What has science actually achieved? A theory of achievement should (1) define what has been achieved, (2) describe the means or methods used in science, and (3) explain how such methods lead to such achievements. Predictive accuracy is one truth-related achievement of science, and there is an explanation of why common scientific practices (of trading off simplicity and fit) tend to increase predictive accuracy. Akaike’s explanation for the success of AIC is limited to interpolative predictive accuracy. But therein lies the strength of the general framework, for it also provides a clear formulation of many open problems of research.