## Information markets vs. opinion pools: An empirical comparison (2005)

### Cached

### Download Links

- [www.eecs.harvard.edu]
- [www.yiling.seas.harvard.edu]
- [www.smeal.psu.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | In Proceedings of the Sixth ACM Conference on Electronic Commerce (EC’05 |

Citations: | 14 - 7 self |

### BibTeX

@INPROCEEDINGS{Chen05informationmarkets,

author = {Yiling Chen and Chao-hsien Chu and Tracy Mullen and David M. Pennock},

title = {Information markets vs. opinion pools: An empirical comparison},

booktitle = {In Proceedings of the Sixth ACM Conference on Electronic Commerce (EC’05},

year = {2005}

}

### OpenURL

### Abstract

In this paper, we examine the relative forecast accuracy of information markets versus expert aggregation. We leverage a unique data source of almost 2000 people’s subjective probability judgments on 2003 US National Football League games and compare with the “market probabilities ” given by two different information markets on exactly the same events. We combine assessments of multiple experts via linear and logarithmic aggregation functions to form pooled predictions. Prices in information markets are used to derive market predictions. Our results show that, at the same time point ahead of the game, information markets provide as accurate predictions as pooled expert assessments. In screening pooled expert predictions, we find that arithmetic average is a robust and efficient pooling function; weighting expert assessments according to their past performance does not improve accuracy of pooled predictions; and logarithmic aggregation functions offer bolder predictions than linear aggregation functions. The results provide insights into the predictive performance of information markets, and the relative merits of selecting among various opinion pooling methods.

### Citations

724 |
Efficient Capital Markets: A Review of Theory and Empirical Work
- Fama
- 1970
(Show Context)
Citation Context ...fied, and can’t guarantee convergence or repeatability.3. HOW INFORMATION MARKETS WORK Much of the enthusiasm for information markets stems from Hayek hypothesis [26] and efficient market hypothesis =-=[15]-=-. Hayek, in his classic critique of central planning in 1940’s, claims that the price system in a competitive market is a very efficient mechanism to aggregate dispersed information among market parti... |

333 |
The use of knowledge in society
- Hayek
- 1945
(Show Context)
Citation Context ...hem are not fixed or completely specified, and can’t guarantee convergence or repeatability.3. HOW INFORMATION MARKETS WORK Much of the enthusiasm for information markets stems from Hayek hypothesis =-=[26]-=- and efficient market hypothesis [15]. Hayek, in his classic critique of central planning in 1940’s, claims that the price system in a competitive market is a very efficient mechanism to aggregate dis... |

314 | How to use expert advice
- Cesa-Bianchi, Freund, et al.
- 1997
(Show Context)
Citation Context ...heir relative prediction accuracy with other belief aggregation methods such as Bayesian approaches. There are also a number of theoretical expert algorithms with proven worst-case performance bounds =-=[10]-=- whose average-case or practical performance would be instructive to investigate. • Whether defining expertise more narrowly can improve predictions of opinion pools. In our analysis, we broadly treat... |

179 |
Combining probability distributions: a critique and an annotated bibliography
- Genest, Zidek
- 1986
(Show Context)
Citation Context ...thods have been proposed to combine expert opinions together and form a single prediction. These belief aggregation methods are called opinion pools, which have been extensively studied in statistics =-=[20, 24, 38]-=-, and management sciences [8, 9, 30, 31], and applied in many domains such as group decision making [29] and risk analysis [12]. With the fast growth of the Internet, information markets have recently... |

174 |
Recovering probabilities distributions from option prices
- Jackwerth, Rubinstein
- 1996
(Show Context)
Citation Context ... all relevant information across traders, hence is the market participants’ consensus expectation about the future value of the security. Empirical evidence supports both hypotheses to a large extent =-=[25, 27, 35]-=-. Thus, when associating the value of a security with the outcome of an uncertain future event, market price, by revealing the consensus expectation of the security value, can indirectly predict the o... |

167 |
Introduction to the Delphi method: techniques and applications
- Linstone, Turoff
- 1975
(Show Context)
Citation Context ...hanging opinions or information, experts can eventually reach an equilibrium where further interaction won’t change their opinions. One of the best known behavioral approaches is the Delphi technique =-=[28]-=-. Typically, this method and its variants do not allow open discussion, but each expert has chance to judge opinions of other experts, and is given feedback. Experts then can reassess their opinions a... |

113 |
Computer-Intensive Methods for Testing Hypotheses : An Introduction. Wiley-Interscience
- Noreen
- 1989
(Show Context)
Citation Context ...an the opinion pools. However, the median values of accuracy measures indicate that LogAll-u and Log-All-w opinion pools are more accurate than all other predictions. We employ the randomization test =-=[32]-=- to study whether the differences in prediction accuracy presented in Table 3 and Table 4 are statistically significant. The basic idea of randomization test is that, by randomly swapping predictions ... |

103 |
Rational expectations and the aggregation of diverse information in laboratory security markets
- Plott, Sunder
- 1988
(Show Context)
Citation Context ... all relevant information across traders, hence is the market participants’ consensus expectation about the future value of the security. Empirical evidence supports both hypotheses to a large extent =-=[25, 27, 35]-=-. Thus, when associating the value of a security with the outcome of an uncertain future event, market price, by revealing the consensus expectation of the security value, can indirectly predict the o... |

86 | Prediction markets
- WOLFERS, ZITZEWITZ
- 2004
(Show Context)
Citation Context ...f a prediction problem can be any value in a continuous interval, we can design a security that pays its holder proportional to the realized value. This kind of security is what Wolfers and Zitzewitz =-=[40]-=- called an index contract. It predicts the expected value of a future outcome. Many other aspects of a future event such as median value of outcome can also be predicted in information markets by desi... |

83 |
Combining probability distributions from experts in risk analysis
- Clemen, Winkler
- 1999
(Show Context)
Citation Context ...opinion pools, which have been extensively studied in statistics [20, 24, 38], and management sciences [8, 9, 30, 31], and applied in many domains such as group decision making [29] and risk analysis =-=[12]-=-. With the fast growth of the Internet, information markets have recently emerged as a promising non-statistical forecasting tool. Information markets (sometimes called prediction markets, idea market... |

68 |
Anatomy of an experimental political stock market
- Forsythe, Nelson, et al.
- 1992
(Show Context)
Citation Context ...information markets predict? Previous research in general shows that information markets are remarkably accurate. The political election markets at IEM predict the election outcomes better than polls =-=[16, 17, 18, 19]-=-. Prices in HSX and FX have been found to give as accurate or more accurate predictions than judgment of individual experts [33, 34, 37]. However, information markets have not been calibrated against ... |

64 |
An introduction to the theory of rational expectations under asymmetric information
- Grossman
- 1981
(Show Context)
Citation Context ... all relevant information across traders, hence is the market participants’ consensus expectation about the future value of the security. Empirical evidence supports both hypotheses to a large extent =-=[25, 27, 35]-=-. Thus, when associating the value of a security with the outcome of an uncertain future event, market price, by revealing the consensus expectation of the security value, can indirectly predict the o... |

44 |
Information aggregation in an experimental market
- Forsythe, Lundholm
- 1990
(Show Context)
Citation Context ...information markets predict? Previous research in general shows that information markets are remarkably accurate. The political election markets at IEM predict the election outcomes better than polls =-=[16, 17, 18, 19]-=-. Prices in HSX and FX have been found to give as accurate or more accurate predictions than judgment of individual experts [33, 34, 37]. However, information markets have not been calibrated against ... |

31 | Extracting collective probabilistic forecasts from web games
- Pennock, Nielsen, et al.
- 2001
(Show Context)
Citation Context ...kets at IEM predict the election outcomes better than polls [16, 17, 18, 19]. Prices in HSX and FX have been found to give as accurate or more accurate predictions than judgment of individual experts =-=[33, 34, 37]-=-. However, information markets have not been calibrated against opinion pools, except for Servan-Schreiber et. al [36], in which the authors compare two information markets against arithmetic average ... |

29 |
Decision analysis expert use
- Morris
- 1974
(Show Context)
Citation Context ...ion methods are called opinion pools, which have been extensively studied in statistics [20, 24, 38], and management sciences [8, 9, 30, 31], and applied in many domains such as group decision making =-=[29]-=- and risk analysis [12]. With the fast growth of the Internet, information markets have recently emerged as a promising non-statistical forecasting tool. Information markets (sometimes called predicti... |

29 | Internet-Based Virtual Stock Markets for Business Forecasting
- Spann, Skiera
(Show Context)
Citation Context ...kets at IEM predict the election outcomes better than polls [16, 17, 18, 19]. Prices in HSX and FX have been found to give as accurate or more accurate predictions than judgment of individual experts =-=[33, 34, 37]-=-. However, information markets have not been calibrated against opinion pools, except for Servan-Schreiber et. al [36], in which the authors compare two information markets against arithmetic average ... |

29 |
The consensus of subjective probability distributions
- Winkler
- 1968
(Show Context)
Citation Context ...re k is a normalization constant to ensure that the pooled opinion is a probability distribution. Other axiomatic pooling methods often are extensions of LinOP [22], LogOP [23], or both [13]. Winkler =-=[39]-=- and Morris [29, 30] establish the early framework of Bayesian aggregation methods. Bayesian approaches assume as if there is a decision maker who has a prior probability distribution over event θ and... |

27 |
actions: A survey on price formation in election stock markets
- Wishes
- 1999
(Show Context)
Citation Context ...information markets predict? Previous research in general shows that information markets are remarkably accurate. The political election markets at IEM predict the election outcomes better than polls =-=[16, 17, 18, 19]-=-. Prices in HSX and FX have been found to give as accurate or more accurate predictions than judgment of individual experts [33, 34, 37]. However, information markets have not been calibrated against ... |

26 |
Forecaster diversity and the benefits of combining forecasts
- Batchelor, R, et al.
- 1995
(Show Context)
Citation Context ...pert opinions together and form a single prediction. These belief aggregation methods are called opinion pools, which have been extensively studied in statistics [20, 24, 38], and management sciences =-=[8, 9, 30, 31]-=-, and applied in many domains such as group decision making [29] and risk analysis [12]. With the fast growth of the Internet, information markets have recently emerged as a promising non-statistical ... |

26 |
Combining expert judgments: A Bayesian approach
- Morris
- 1977
(Show Context)
Citation Context ...pert opinions together and form a single prediction. These belief aggregation methods are called opinion pools, which have been extensively studied in statistics [20, 24, 38], and management sciences =-=[8, 9, 30, 31]-=-, and applied in many domains such as group decision making [29] and risk analysis [12]. With the fast growth of the Internet, information markets have recently emerged as a promising non-statistical ... |

24 | The real power of artificial markets
- Pennock, Lawrence, et al.
(Show Context)
Citation Context ...kets at IEM predict the election outcomes better than polls [16, 17, 18, 19]. Prices in HSX and FX have been found to give as accurate or more accurate predictions than judgment of individual experts =-=[33, 34, 37]-=-. However, information markets have not been calibrated against opinion pools, except for Servan-Schreiber et. al [36], in which the authors compare two information markets against arithmetic average ... |

23 |
Prediction markets: Does money matter
- Servan-Schreiber, Wolfers, et al.
- 2004
(Show Context)
Citation Context ...accurate or more accurate predictions than judgment of individual experts [33, 34, 37]. However, information markets have not been calibrated against opinion pools, except for Servan-Schreiber et. al =-=[36]-=-, in which the authors compare two information markets against arithmetic average of expert opinions. Since information markets, in nature, offer an adaptive and selforganized mechanism to aggregate o... |

20 |
Aggregating subjective forecasts: Some empirical results
- Ashton, Ashton
- 1985
(Show Context)
Citation Context ...pert opinions together and form a single prediction. These belief aggregation methods are called opinion pools, which have been extensively studied in statistics [20, 24, 38], and management sciences =-=[8, 9, 30, 31]-=-, and applied in many domains such as group decision making [29] and risk analysis [12]. With the fast growth of the Internet, information markets have recently emerged as a promising non-statistical ... |

15 |
Characterization of externally Bayesian pooling operators
- Genest, McConway, et al.
- 1986
(Show Context)
Citation Context ...=k n∏ pi(θ) wi , (2) i=1 where k is a normalization constant to ensure that the pooled opinion is a probability distribution. Other axiomatic pooling methods often are extensions of LinOP [22], LogOP =-=[23]-=-, or both [13]. Winkler [39] and Morris [29, 30] establish the early framework of Bayesian aggregation methods. Bayesian approaches assume as if there is a decision maker who has a prior probability d... |

13 |
Predicting the Future
- Chen, Fine, et al.
(Show Context)
Citation Context ...ll the predictions of opinion pools be more accurate? • The possibility of combining information markets with other forecasting methods to achieve better prediction accuracy. Chen, Fine, and Huberman =-=[11]-=- use an information market to determine the risk attitude of participants, and then perform a nonlinear aggregation of their predictions based on their risk attitudes. The nonlinear aggregation mechan... |

8 |
A conflict between two axioms for combining subjective distributions
- Genest
- 1984
(Show Context)
Citation Context ...ironments. Each approach has its pros and cons. Axiomatic approaches are easy to use. But they don’t have a normative basis to choose weights. In addition, several impossibility results (e.g., Genest =-=[21]-=-) show that no aggregation function can satisfy all desired properties of an opinion pool, unless the pooled opinion degenerates to a single individual opinion, which effectively implies a dictator. B... |

8 |
An axiomatic approach to expert resolution
- Morris
- 1983
(Show Context)
Citation Context |

7 |
Experts in Uncertainty: Opinion and Subjective Probability in Science
- Cook
- 1991
(Show Context)
Citation Context ... , (2) i=1 where k is a normalization constant to ensure that the pooled opinion is a probability distribution. Other axiomatic pooling methods often are extensions of LinOP [22], LogOP [23], or both =-=[13]-=-. Winkler [39] and Morris [29, 30] establish the early framework of Bayesian aggregation methods. Bayesian approaches assume as if there is a decision maker who has a prior probability distribution ov... |

7 |
Pooling operators with the marginalization property
- Genest
- 1984
(Show Context)
Citation Context ...tions: p(θ) =k n∏ pi(θ) wi , (2) i=1 where k is a normalization constant to ensure that the pooled opinion is a probability distribution. Other axiomatic pooling methods often are extensions of LinOP =-=[22]-=-, LogOP [23], or both [13]. Winkler [39] and Morris [29, 30] establish the early framework of Bayesian aggregation methods. Bayesian approaches assume as if there is a decision maker who has a prior p... |

7 |
M: Bayesian aggregation
- West
- 1984
(Show Context)
Citation Context ...thods have been proposed to combine expert opinions together and form a single prediction. These belief aggregation methods are called opinion pools, which have been extensively studied in statistics =-=[20, 24, 38]-=-, and management sciences [8, 9, 30, 31], and applied in many domains such as group decision making [29] and risk analysis [12]. With the fast growth of the Internet, information markets have recently... |

2 |
Group Techniques for Program Planners: A Guide to
- Delbecq, Ven, et al.
- 1975
(Show Context)
Citation Context ...back. Experts then can reassess their opinions and repeat the process until a consensus or a smaller spread of opinions is achieved. Some other behavioral methods, such as the Nominal Group technique =-=[14]-=-, promote open discussions in controlled environments. Each approach has its pros and cons. Axiomatic approaches are easy to use. But they don’t have a normative basis to choose weights. In addition, ... |

2 |
Forecasting elections: A market alternative topolls.InT.R.Palfrey,editor,Contemporary Laboratory Experiments
- Forsythe, Nelson, et al.
- 1991
(Show Context)
Citation Context |

2 |
Group consensus probability distributions: acriticalsurvey.Bayesian Statistics
- French
- 1985
(Show Context)
Citation Context ...thods have been proposed to combine expert opinions together and form a single prediction. These belief aggregation methods are called opinion pools, which have been extensively studied in statistics =-=[20, 24, 38]-=-, and management sciences [8, 9, 30, 31], and applied in many domains such as group decision making [29] and risk analysis [12]. With the fast growth of the Internet, information markets have recently... |