Results 11  20
of
123
Adaptive Polling for Information Aggregation
"... The flourishing of online labor markets such as Amazon Mechanical Turk (MTurk) makes it easy to recruit many workers for solving small tasks. We study whether information elicitation and aggregation over a combinatorial space can be achieved by integrating small pieces of potentially imprecise infor ..."
Abstract

Cited by 8 (1 self)
 Add to MetaCart
(Show Context)
The flourishing of online labor markets such as Amazon Mechanical Turk (MTurk) makes it easy to recruit many workers for solving small tasks. We study whether information elicitation and aggregation over a combinatorial space can be achieved by integrating small pieces of potentially imprecise information, gathered from a large number of workers through simple, oneshot interactions in an online labor market. We consider the setting of predicting the ranking of n competing candidates, each having a hidden underlying strength parameter. At each step, our method estimates the strength parameters from the collected pairwise comparison data and adaptively chooses another pairwise comparison question for the next recruited worker. Through an MTurk experiment, we show that the adaptive method effectively elicits and aggregates information, outperforming a naïve method using a random pairwise comparison question at each step. 1
Prediction without Markets
 Association for Computing Machinery
, 2010
"... Citing recent successes in forecasting elections, movies, products, and other outcomes, prediction market advocates call for widespread use of marketbased methods for government and corporate decision making. Though theoretical and empirical evidence suggests that markets do often outperform altern ..."
Abstract

Cited by 7 (1 self)
 Add to MetaCart
(Show Context)
Citing recent successes in forecasting elections, movies, products, and other outcomes, prediction market advocates call for widespread use of marketbased methods for government and corporate decision making. Though theoretical and empirical evidence suggests that markets do often outperform alternative mechanisms, less attention has been paid to the magnitude of improvement. Here we compare the performance of prediction markets to conventional methods of prediction, namely polls and statistical models. Examining thousands of sporting and movie events, we find that the relative advantage of prediction markets is surprisingly small, as measured by squared error, calibration, and discrimination. Moreover, these domains also exhibit remarkably steep diminishing returns to information, with nearly all the predictive power captured by only two or three parameters. As policy makers consider adoption of prediction markets, costs should be weighed against potentially modest benefits.
Supervised Aggregation of Classifiers using Artificial Prediction Markets
"... Prediction markets are used in real life to predict outcomes of interest such as presidential elections. In this work we introduce a mathematical theory for Artificial Prediction Markets for supervised classifier aggregation and probability estimation. We introduce the artificial prediction market a ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
(Show Context)
Prediction markets are used in real life to predict outcomes of interest such as presidential elections. In this work we introduce a mathematical theory for Artificial Prediction Markets for supervised classifier aggregation and probability estimation. We introduce the artificial prediction market as a novel way to aggregate classifiers. We derive the market equations to enforce total budget conservation, show the market price uniqueness and give efficient algorithms for computing it. We show how to train the market participants by updating their budgets using training examples. We introduce classifier specialization as a new differentiating characteristic between classifiers. Finally, we present experiments using random decision rules as specialized classifiers and show that the prediction market consistently outperforms Random Forest on real and synthetic data of varying degrees of difficulty. 1.
Information elicitation for decision making
, 2011
"... Proper scoring rules, particularly when used as the basis for a prediction market, are powerful tools for eliciting and aggregating beliefs about events such as the likely outcome of an election or sporting event. Such scoring rules incentivize a single agent to reveal her true beliefs about the eve ..."
Abstract

Cited by 6 (4 self)
 Add to MetaCart
(Show Context)
Proper scoring rules, particularly when used as the basis for a prediction market, are powerful tools for eliciting and aggregating beliefs about events such as the likely outcome of an election or sporting event. Such scoring rules incentivize a single agent to reveal her true beliefs about the event. Othman and Sandholm [16] introduced the idea of a decision rule to examine these problems in contexts where the information being elicited is conditional on some decision alternatives. For example, “What is the probability having ten million viewers if we choose to air new television show X? What if we choose Y? ” Since only one show can actually air in a slot, only the results under the chosen alternative can ever be observed. Othman and Sandholm developed proper scoring rules (and thus decision markets) for a single, deterministic decision rule: always select the the action with the greatest probability of success. In this work we significantly generalize their results, developing scoring rules for other deterministic decision rules, randomized decision rules, and situations where there may be more than two outcomes (e.g. less than a million viewers, more than one but less than ten, or more than ten million).
Efficient market making via convex optimization, and a connection to online learning
 ACM Transactions on Economics and Computation. To Appear
, 2012
"... We propose a general framework for the design of securities markets over combinatorial or infinite state or outcome spaces. The framework enables the design of computationally efficient markets tailored to an arbitrary, yet relatively small, space of securities with bounded payoff. We prove that any ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
(Show Context)
We propose a general framework for the design of securities markets over combinatorial or infinite state or outcome spaces. The framework enables the design of computationally efficient markets tailored to an arbitrary, yet relatively small, space of securities with bounded payoff. We prove that any market satisfying a set of intuitive conditions must price securities via a convex cost function, which is constructed via conjugate duality. Rather than deal with an exponentially large or infinite outcome space directly, our framework only requires optimization over a convex hull. By reducing the problem of automated market making to convex optimization, where many efficient algorithms exist, we arrive at a range of new polynomialtime pricing mechanisms for various problems. We demonstrate the advantages of this framework with the design of some particular markets. We also show that by relaxing the convex hull we can gain computational tractability without compromising the market institution’s bounded budget. Although our framework was designed with the goal of deriving efficient automated market makers for markets with very large outcome spaces, this framework also provides new insights into the relationship between market design and machine learning, and into the complete market setting. Using our framework, we illustrate the mathematical parallels between cost function based markets and online learning and establish a correspondence between cost function based markets and market scoring rules for complete markets. 1
An introduction to artificial prediction markets for classification. arXiv:1102.1465
, 2011
"... ..."
(Show Context)
Decision Markets With Good Incentives
"... Abstract. Decision and prediction markets are designed to determine the likelihood of future events; prediction markets predict what will happen, and decision markets predict the results of a choice, or what would happen. Both allow multiple participants to review and make predictions, and participa ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
(Show Context)
Abstract. Decision and prediction markets are designed to determine the likelihood of future events; prediction markets predict what will happen, and decision markets predict the results of a choice, or what would happen. Both allow multiple participants to review and make predictions, and participants are typically scored for improving the accuracy of the market’s prediction. Previous work has demonstrated prediction markets can reward accuracy improvements, as can a single participant informing a decision. We construct and characterize decision markets where all participants are scored for improving the market’s accuracy. These markets require the decision maker always risk taking an action at random, and reducing this risk increases its potential loss. We also relate these decision markets to sets of prediction markets, demonstrating a correspondence between their perfect Bayesian equilibria. 1
Security Economics and European Policy
, 2008
"... In September 2007, we were awarded a contract by the European Network and Information Security Agency (ENISA) to investigate failures in the market for secure electronic communications within the European Union, and come up with policy recommendations. In the process, we spoke to a large number of s ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
(Show Context)
In September 2007, we were awarded a contract by the European Network and Information Security Agency (ENISA) to investigate failures in the market for secure electronic communications within the European Union, and come up with policy recommendations. In the process, we spoke to a large number of stakeholders, and held a consultative meeting in December 2007 in Brussels to present draft proposals. This established that almost all of our proposals have wide stakeholder support. The formal outcome of our work was a detailed report, ‘Security Economics and the Internal Market’, that is due to be published by ENISA. This paper is a much abridged version (about half the length): in it, we present the recommendations we made, and then a summary of our reasoning. By way of disclaimer, we state that these recommendations are our own and do not necessarily reflect the policy of ENISA or any other European institution. The background should be familiar enough. The direct cost to Europe of electronic crime, including both losses and protective measures, is measured in billions of euros; and growing public concerns about information security hinder the development of both markets and public services, causing even greater indirect costs. For example, while we
Mechanism design for the truthful elicitation of costly probabilistic estimates in distributed information systems
 Artif. Intell
"... This paper reports on the design of a novel twostage mechanism, based on strictly proper scoring rules, that allows a centre to acquire a costly forecast of a future event (such as a meteorological phenomenon) or a probabilistic estimate of a specific parameter (such as the quality of an expected s ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
(Show Context)
This paper reports on the design of a novel twostage mechanism, based on strictly proper scoring rules, that allows a centre to acquire a costly forecast of a future event (such as a meteorological phenomenon) or a probabilistic estimate of a specific parameter (such as the quality of an expected service), with a specified minimum precision, from one or more agents. In the first stage, the centre elicits the agents ’ true costs and identifies the agent that can provide an estimate of the specified precision at the lowest cost. Then, in the second stage, the centre uses an appropriately scaled strictly proper scoring rule to incentivise this agent to generate the estimate with the required precision, and to truthfully report it. In particular, this is the first mechanism that can be applied to settings in which the centre has no knowledge about the actual costs involved in the generation an agents ’ estimates and also has no external means of evaluating the quality and accuracy of the estimates it receives. En route to this mechanism, we first consider a setting in which any single agent can provide an estimate of the required precision, and the centre can evaluate this estimate by comparing it with the outcome which is observed at a later stage. This mechanism is then extended, so that it can be applied in a setting where the agents ’ different capabilities are reflected in the maximum precision of the estimates that they can provide, potentially requiring the centre to select mul
The future of humanity
 In: Olsen J‐KB, Selinger E, Riis S (eds). New Waves in Philosophy of Technology
, 2009
"... [Reprinted in the journal Geopolitics, History, and International Relations, forthcoming] The future of humanity is often viewed as a topic for idle speculation. Yet our beliefs and assumptions on this subject matter shape decisions in both our personal lives and public policy – decisions that have ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
(Show Context)
[Reprinted in the journal Geopolitics, History, and International Relations, forthcoming] The future of humanity is often viewed as a topic for idle speculation. Yet our beliefs and assumptions on this subject matter shape decisions in both our personal lives and public policy – decisions that have very real and sometimes unfortunate consequences. It is therefore practically important to try to develop a realistic mode of futuristic thought about big picture questions for humanity. This paper sketches an overview of some recent attempts in this direction, and it offers a brief discussion of four families of scenarios for humanity’s future: extinction, recurrent collapse, plateau, and posthumanity. The future of humanity as an inescapable topic In one sense, the future of humanity comprises everything that will ever happen to any human being, including what you will have for breakfast next Thursday and all the scientific discoveries that will be made next year. In that sense, it is hardly reasonable to think of the future of humanity as a topic: it is too big and too diverse to be addressed as a whole in a single essay, monograph, or even 100volume book series. It is made into a topic by way of