Abstract not found

Please do not cite this version.

The growing literature on learning in games has produced various results on the predictive success of learning theories. These results, however, were based on various methods of comparison. The present paper uses experimental data on a set of four games in order to check on the robustness of rankings among learning rules across measures. We characterise measures along three dimensions: (i) the scoring rule, (ii) the method of comparison, and (iii) the definition of observations and apply all thus defined measures to 12 learning rules.

We study principles and methods for task routing that aim to harness people’s abilities to jointly contribute to a task and to route tasks to others who can provide further contributions. In the particular context of prediction tasks, the goal is to efficiently obtain accurate probability assessments for an event of interest. We introduce routing scoring rules for promoting collaborative behavior, that bring truthfully contributing information and optimally routing tasks into a Perfect Bayesian Equilibrium under common knowledge about agents ’ abilities. However, for networks where agents only have local knowledge about other agents ’ abilities, optimal routing requires complex reasoning over the history and future routing decisions of users outside of local neighborhoods. Avoiding this, we introduce a class of local routing rules that isolate simple routing decisions in equilibrium, while still promoting effective routing decisions. We present simulation results that show that following routing decisions induced by local routing rules lead to efficient information aggregation.

We wish to thank the staff and facilities of the Center for Behavioral Decision Research and the

Scoring rules assess the quality of probabilistic forecasts, by assigning a numerical score based on the predictive distribution and on the event or value that materializes. A scoring rule is proper if it encourages truthful reporting. It is local of order λ if the score depends on the predictive density only through its value and its derivatives of order up to λ at the observation. Previously, only a single local proper scoring rule had been known, namely the logarithmic score, which is local of order λ = 0. Here we introduce the Fisher score, which is a local proper scoring rule of order λ = 2. It relates to the Fisher information in the same way that the logarithmic score relates to the Kullback-Leibler information. The convex cone generated by the logarithmic score and the Fisher score exhausts the class of the local proper scoring rules of order λ ≤ 2, up to equivalence and regularity conditions. In a data example, we use local and non-local proper scoring rules to assess statistically postprocessed ensemble weather forecasts. Finally, we develop a multivariate version of the Fisher score. 1

Several recent studies in experimental economics have tried to measure beliefs of subjects engaged in strategic games with other subjects. Using data from one such study (Nyarko-Schotter, 2002) we conduct an experiment where our experienced subjects observe early rounds of strategy choices from that study and are given monetary incentives to report forecasts of choices in later rounds. We elicit beliefs using three different scoring rules: linear, logarithmic, and quadratic. There are differences between the elicited beliefs under quadratic and logarithmic scoring rules in spite of both being proper scoring rules. The (improper) linear scoring rule frequently elicits boundary forecasts as theory predicts, and is poorly calibrated. We compare the forecasts of our trained observers to forecasts of the actual players in the Nyarko-Schotter experiment and identify several differences. There was a significant positive correlation between observer forecasts and the choice behavior in the game under both proper scoring rules, but no significant correlation between the players ’ own forecasts and the actual play. This raises doubts about whether beliefs can be reliably elicited from players who simultaneously have a stake in the target of their forecast, in

Theoretical models of information asymmetry have identified a tradeoff between the desire to learn and the desire to prevent an opponent from learning private information. This paper reports a laboratory experiment that investigates if actual bidders account for this tradeoff, using a sequential procurement auction with private cost information and varying information revelation policies. Specifically, the Complete Information Policy, where all submitted bids are revealed between auctions, is compared against the Incomplete Information Policy, where only the winning bid is revealed. The experimental results are largely consistent with the theoretical predictions. For example, bidders pool with other types to prevent an opponent from learning significantly more often under a Complete Information Policy. Also as predicted, the procurer pays less when employing an Incomplete Information Policy only when the market is highly competitive. Bids are usually more aggressive than the risk neutral quantitative prediction, which is usually consistent with risk aversion.

How do people learn to allocate resources? To answer this question, two major learning models are compared, each incorporating different learning principles. One is a global search model, which assumes that allocations are made probabilistically based on expectations that are formed through the entire history of past decisions. The second is a local adaptation model, which assumes that allocations are made by comparing the present decision to the most successful decision up to that point, ignoring all other past decisions. The models ’ predictions are tested in two studies in which participants repeatedly allocated a capital resource to three financial assets. In both studies substantial learning effects occurred although the optimal allocation was often not found. In particular, many participants got trapped by a local rather than a global maximum. From the calibrated models of Study 1 a priori predictions could be produced and tested in Study 2. This generalization test demonstrated that the local adaptation model provides a better account of learning in resource allocation tasks than the global search model.

Erev, Ert, and Roth organized three choice prediction competitions focused on three related choice tasks: one shot decisions from description (decisions under risk), one shot decisions from experience, and repeated decisions from experience. Each competition was based on two experimental datasets: An estimation dataset, and a competition dataset. The studies that generated the two datasets used the same methods and subject pool, and examined decision problems randomly selected from the same distribution. After collecting the experimental data to be used for estimation, the organizers posted them on the Web, together with their fit with several baseline models, and challenged other researchers to compete to predict the results of the second (competition) set of experimental sessions. Fourteen teams responded to the challenge: the last seven authors of this paper are members of the winning teams. The results highlight the robustness of the difference between decisions from description and decisions from experience. The best predictions of decisions from descriptions were obtained with a stochastic variant of prospect theory assuming that the sensitivity to the weighted values decreases with the