Humans and animals make inferences about the world under limited time and knowledge. In contrast, many models of rational inference treat the mind as a Laplacean Demon, equipped with unlimited time, knowledge, and computational might. Following H. Simon’s notion of satisficing, the authors have proposed a family of algorithms based on a simple psychological mechanism: one reason decision making. These fast and frugal algorithms violate fundamental tenets of classical rationality: They neither look up nor integrate all information. By computer simulation, the authors held a competition between the satisficing “Take The Best ” algorithm and various “rational ” inference procedures (e.g., multiple regression). The Take The Best algorithm matched or outperformed all competitors in inferential speed and accuracy. This result is an existence proof that cognitive mechanisms capable of successful performance in the real world do not need to satisfy the classical norms of rational inference. Organisms make inductive inferences. Darwin (1872/1965) observed that people use facial cues, such as eyes that waver and lids that hang low, to infer a person’s guilt. Male toads, roaming through swamps at night, use the pitch of a rival’s croak to infer its size when deciding whether to fight (Krebs & Davies, 1987). Stock brokers must make fast decisions about which of several stocks to trade or invest when only limited information is available. The list goes on. Inductive

A briefer version of this paper with the same name has been published in

Research on people’s confidence in their general knowledge has to date produced two fairly stable effects, many inconsistent results, and no comprehensive theory. We propose such a comprehensive framework, the theory of probabilistic mental models (PMM theory). The theory (a) explains both the overconfidence effect (mean confidence is higher than percentage of answers correct) and the hard-easy effect (overconfidence increases with item difficulty) reported in the literature and (b) predicts conditions under which both effects appear, disappear, or invert. In addition, (c) it predicts a new phenomenon, the confidence-frequency effect, a systematic difference between a judgment of confidence in a single event (i.e., that any given answer is correct) and a judgment of the frequency of correct answers in the long run. Two experiments are reported that support PMM theory by confirming these predictions, and several apparent anomalies reported in the literature are explained and integrated into the present framework. Do people think they know more than they really do? In the last 15 years, cognitive psychologists have amassed a large and apparently damning body of experimental evidence on overconfidence in knowledge, evidence that is in turn part of an even larger and more damning literature on socalled cognitive biases. The cognitive bias research claims that people are naturally prone to making mistakes in reasoning and memory, including the mistake of overestimating their knowledge.

Confirmation bias, as the term is typically used in the psychological literature, connotes the seeking or interpreting of evidence in ways that are partial to existing beliefs, expectations, or a hypothesis in hand. The author reviews evidence of such a bias in a variety of guises and gives examples of its operation in several practical contexts. Possible explanations are considered, and the question of its utility or disutility is discussed. When men wish to construct or support a theory, how they torture facts into their service! (Mackay, 1852/ 1932, p. 552) Confirmation bias is perhaps the best known and most widely accepted notion of inferential error to come out of the literature on human reasoning. (Evans, 1989, p. 41) If one were to attempt to identify a single problematic aspect of human reasoning that deserves attention above all others, the confirmation bias would have to be among the candidates for consideration. Many have written about this bias, and it appears to be sufficiently strong and pervasive that one is led to wonder whether the bias, by itself, might account for a significant fraction of the disputes, altercations, and misunderstandings that occur among individuals, groups, and nations. Confirmation bias has been used in the psychological literature to refer to a variety of phenomena. Here I take the term to represent a generic concept that subsumes several more specific ideas that connote the inappropriate bolstering of hypotheses or beliefs whose truth is in question.

This article discusses the history underlying the new literature, describes some of what has been learned thus far, and looks ahead towards making further progress

This article argues that (a) ego, or self, is an organization of knowledge, (b) ego is characterized by cognitive biases strikingly analogous to totalitarian information-control strategies, and (c) these totalitarian-ego biases junction to preserve organization in cognitive structures. Ego's cognitive biases are egocentricity (self as the focus of knowledge), "beneffectance" (perception of responsibility for desired, but not undesired, outcomes), and cognitive conservatism (resistance to cognitive change). In addition to being pervasively evident in recent studies of normal human cognition, these three biases are found in actively functioning, higher level organizations of knowledge, perhaps best exemplified by theoretical paradigms in science. The thesis that egocentricity, beneffectance, and

Elicitation is a key task for subjectivist Bayesians. While skeptics hold that it cannot (or perhaps should not) be done, in practice it brings statisticians closer to their clients and subjectmatter-expert colleagues. This paper reviews the state-of-the-art, reflecting the experience of statisticians informed by the fruits of a long line of psychological research into how people represent uncertain information cognitively, and how they respond to questions about that information. In a discussion of the elicitation process, the first issue to address is what it means for an elicitation to be successful, i.e. what criteria should be employed? Our answer is that a successful elicitation faithfully represents the opinion of the person being elicited. It is not necessarily “true ” in some objectivistic sense, and cannot be judged that way. We see elicitation as simply part of the process of statistical modeling. Indeed in a hierarchical model it is ambiguous at which point the likelihood ends and the prior begins. Thus the same kinds of judgment that inform statistical modeling in general also inform elicitation of prior distributions.

Please do not quote or cite without permission of the authors Overconfidence in interval estimates 2 Many studies over the last several decades have found that people are generally overconfident about the accuracy of their knowledge. This generalization has been overturned by a number of recent, carefully controlled studies. These studies show little or no overall bias when judges express confidence in a choice between two alternative answers to a question. Apparent overconfidence is due primarily to unsystematic error in judgments, combined with an unrepresentative selection of task items. However, Klayman et al. (1999), found that substantial overconfidence persisted under equivalently controlled conditions with a different type of confidence judgment. When judges are asked to provide intervals such that they are x % sure the correct answer is within the interval, the answer falls inside their interval much less than x % of the time. The present paper shows that, although unsystematic judgmental error may contribute to overconfidence, subjective confidence intervals are indeed systematically too narrow—

The average probability estimate of J> 1 judges is generally better than its components. Two studies test 3 predictions regarding averaging that follow from theorems based on a cognitive model of the judges and idealizations of the judgment situation. Prediction 1 is that the average of conditionally pairwise independent estimates will be highly diagnostic, and Prediction 2 is that the average of dependent estimates (differing only by independent error terms) may be well calibrated. Prediction 3 contrasts between- and within-subject averaging. Results demonstrate the predictions ' robustness by showing the extent to which they hold as the information conditions depart from the ideal and as J increases. Practical consequences are that (a) substantial improvement can be obtained with as few as 2- 6 judges and (b) the decision maker can estimate the nature of the expected improvement by considering the information conditions. On many occasions, experts are required to provide decision makers or policymakers with subjective probability estimates of uncertain events (Morgan & Henrion, 1990). The extensive literature (e.g., Harvey, 1997; McClelland & Bolger, 1994) on the topic shows that in general, but with clear exceptions, subjective

We propose a theory that relates perceived evidence to numerical probability judgment. The most successful prior account of this relation is Support Theory, advanced in Tversky and Koehler (1994). Support Theory, however, implies additive probability estimates for binary partitions. In contrast, superadditivity has been documented in Macchi, Osherson and Krantz (1999), and both sub- and superadditivity appear in the experiments reported here. Nonadditivity suggests asymmetry in the processing of focal and nonfocal hypotheses, even within binary partitions. We extend Support Theory by revising its basic equation to allow such asymmetry, and compare the two equations' ability to predict numerical assessments of probability from scaled estimates of evidence for and against a given proposition. Both between- and within-subject experimental designs are employed for this purpose. We find that the revised equation is more accurate than the original Support Theory equation. The implications of...