Predicting the outcomes of future events is a challenging problem for which a variety of solution methods have been explored and attempted. We present an empirical comparison of a variety of online and offline adaptive algorithms for aggregating experts ’ predictions of the outcomes of five years of US National Football League games (1319 games) using expert probability elicitations obtained from an Internet contest called ProbabilitySports. We find that it is difficult to improve over simple averaging of the predictions in terms of prediction accuracy, but that there is room for improvement in quadratic loss. Somewhat surprisingly, a Bayesian estimation algorithm which estimates the variance of each expert’s prediction exhibits the most consistent superior performance over simple averaging among our collection of algorithms. 1

In this paper, we examine the relative forecast accuracy of information markets versus expert aggregation. We leverage a unique data source of almost 2000 people’s subjective probability judgments on 2003 US National Football League games and compare with the “market probabilities ” given by two different information markets on exactly the same events. We combine assessments of multiple experts via linear and logarithmic aggregation functions to form pooled predictions. Prices in information markets are used to derive market predictions. Our results show that, at the same time point ahead of the game, information markets provide as accurate predictions as pooled expert assessments. In screening pooled expert predictions, we find that arithmetic average is a robust and efficient pooling function; weighting expert assessments according to their past performance does not improve accuracy of pooled predictions; and logarithmic aggregation functions offer bolder predictions than linear aggregation functions. The results provide insights into the predictive performance of information markets, and the relative merits of selecting among various opinion pooling methods.

Ensuring sufficient liquidity is one of the key challenges for designers of prediction markets. Various market making algorithms have been proposed in the literature and deployed in practice, but there has been little effort to evaluate their benefits and disadvantages in a systematic manner. We introduce a novel experimental design for comparing market structures in live trading that ensures fair comparison between two different microstructures with the same trading population. Participants trade on outcomes related to a two-dimensional random walk that they observe on their computer screens. They can simultaneously trade in two markets, corresponding to the independent horizontal and vertical random walks. We use this experimental design to compare the popular inventory-based logarithmic market scoring rule (LMSR) market maker and a new information based Bayesian market maker (BMM). Our experiments reveal that BMM can offer significant benefits in terms of price stability and expected loss when controlling for liquidity; the caveat is that, unlike LMSR, BMM does not guarantee bounded loss. Our investigation also elucidates some general properties of market makers in prediction markets. In particular, there is an inherent tradeoff between adaptability to market shocks and convergence during market equilibrium. 1

Citing recent successes in forecasting elections, movies, products, and other outcomes, prediction market advocates call for widespread use of market-based methods for government and corporate decision making. Though theoretical and empirical evidence suggests that markets do often outperform alternative mechanisms, less attention has been paid to the magnitude of improvement. Here we compare the performance of prediction markets to conventional methods of prediction, namely polls and statistical models. Examining thousands of sporting and movie events, we find that the relative advantage of prediction markets is surprisingly small, as measured by squared error, calibration, and discrimination. Moreover, these domains also exhibit remarkably steep diminishing returns to information, with nearly all the predictive power captured by only two or three parameters. As policy makers consider adoption of prediction markets, costs should be weighed against potentially modest benefits.

Abstract. We present Turing Trade, a web-based game that is a hybrid of a Turing test and a prediction market. In this game, there is a mystery conversation partner, the “target, ” who is trying to appear human, but may in reality be either a human or a bot. There are multiple judges (or “bettors”), who interrogate the target in order to assess whether it is a human or a bot. Throughout the interrogation, each bettor bets on the nature of the target by buying or selling human (or bot) securities, which pay out if the target is a human (bot). The resulting market price represents the bettors ’ aggregate belief that the target is a human. This game offers multiple advantages over standard variants of the Turing test. Most significantly, our game gathers much more fine-grained data, since we obtain not only the judges ’ final assessment of the target’s humanity, but rather the entire progression of their aggregate belief over time. This gives us the precise moments in conversations where the target’s response caused a significant shift in the aggregate belief, indicating that the response was decidedly human or unhuman. An additional benefit is that (we believe) the game is more enjoyable to participants than a standard Turing test. This is important because otherwise, we will fail to collect significant amounts of data. In this paper, we describe in detail how Turing Trade works, exhibit some example logs, and analyze how well Turing Trade functions as a prediction market by studying the calibration and sharpness of its forecasts (from real user data). Key words: prediction markets, Turing tests, games with a purpose, deployed webbased applications, using points as an artificial currency 1

Abstract—The ideals of the eighteenth century’s Age of Enlightenment are the foundation of modern democracies. The era was characterized by thinkers who promoted progressive social reforms that opposed the long-established aristocracies and monarchies of the time. Prominent examples of such reforms include the establishment of inalienable human rights, selfgoverning republics, and market capitalism. Twenty-first century democratic nations can benefit from revisiting the systems developed during the Enlightenment and reframing them within the techno-social context of the Information Age. This article explores the application of social algorithms that make use of Thomas Paine’s (English: 1737–1809) representatives, Adam Smith’s (Scottish: 1723–1790) self-interested actors, and Marquis de Condorcet’s (French: 1743–1794) optimal decision making groups. It is posited that technology-enabled social algorithms can better realize the ideals articulated during the Enlightenment. Index Terms—collective decision making, computational governance, e-participation, e-democracy, computational social choice theory. I.

The main objective of this research is to use prediction markets as negotiation agents, for supporting R&D portfolio management. To support this research, we iteratively designed, developed, operated and evaluated several prototypes. We start by presenting the weaknesses of the current techniques for managing R&D portfolio. Then, we intend to demonstrate that prediction markets correct these weaknesses in R&D portfolio management. Furthermore, following a design science paradigm, we illustrate the design of our artifacts using build-andevaluate loops supported with a field study, which consisted in operating the prediction markets in different settings. 1.

There is significant experimental evidence that prediction markets are efficient mechanisms for aggregating information and are more accurate in forecasting events than traditional forecasting methods, such as polls. Interpretation of prediction market prices as probabilities has been extensively studied in the literature. However there is little research on the volatility of prediction market prices. Given that volatility is fundamental in estimating significance of price movements, it is important to have a better understanding of the volatility of the contract prices. This paper presents a model of a prediction market with binary payoff on a competitive event involving two parties. In our model, each party has a latent underlying “ability” process that describes its ability to win and evolves as an Ito diffusion. We show that, if the prediction market for this event is efficient and unbiased, the price of the corresponding contract also follows a diffusion and its instantaneous volatility is a function of the current contract price and its time to expiration. In the experimental section, we validate our model on a set of InTrade prediction markets and show that our model is consistent with the observed volatility of contract returns. Our model also outperforms existing volatility models in predicting future contract volatility from historical price data. To demonstrate the practical value of our model, we apply it to pricing options on prediction market contracts, such as those recently introduced by InTrade. Other potential applications of this model include detection of significant market moves and improving forecast standard errors.

We propose a model of prediction markets where participants are biased according to their social relationships. We relax the standard assumption of complete rationality and adopt an arguably more realistic model where agents are disproportionally influenced by their neighbors in a social network. We conduct extensive agent-based simulations of our model. We find that prices in prediction markets remain accurate even when participants are biased and irrational. Moreover, accuracy is robust to changes in many factors, including how individuals are motivated to participate in the market, the way that individuals use public information, individual utility functions, the topology of the social network, and the strength of social influences. Our model can explain the high volume of trade often observed in speculative markets that is hard or impossible to explain under standard market rationality assumptions. Our model can also explain the documented ability of prediction markets to succeed even in the face of biased and irrational participants.

Problems with scoring