Results 1  10
of
20
Calibrated Learning and Correlated Equilibrium
 Games and Economic Behavior
, 1996
"... Suppose two players meet each other in a repeated game where: 1. each uses a learning rule with the property that it is a calibrated forecast of the others plays, and 2. each plays a best response to this forecast distribution. ..."
Abstract

Cited by 90 (5 self)
 Add to MetaCart
Suppose two players meet each other in a repeated game where: 1. each uses a learning rule with the property that it is a calibrated forecast of the others plays, and 2. each plays a best response to this forecast distribution.
Asymptotic calibration
 Biometrika
, 1998
"... Can we forecast the probability of an arbitrary sequence of events happening so that the stated probability of an event happening is close to its empirical probability? We can view this prediction problem as a game played against nature, where at the beginning of the game Nature picks a data sequenc ..."
Abstract

Cited by 74 (4 self)
 Add to MetaCart
Can we forecast the probability of an arbitrary sequence of events happening so that the stated probability of an event happening is close to its empirical probability? We can view this prediction problem as a game played against nature, where at the beginning of the game Nature picks a data sequence and the forecaster picks a forecasting algorithm. If the forecaster is not allowed to randomize, then Nature win; there will always be data for which the forecaster does poorly. This paper shows that, if the forecaster can randomize, the forecaster wins in the sense that the forecasted probabilities and the empirical probabilities can be made arbitrarily close to each other.
Deterministic calibration and Nash equilibrium
 Proceedings of the Seventeenth Annual Conference on Learning Theory, volume 3120 of Lecture Notes in Computer Science
, 2004
"... Abstract. We provide a natural learning process in which the joint frequency of empirical play converges into the set of convex combinations of Nash equilibria. In this process, all players rationally choose their actions using a public prediction made by a deterministic, weakly calibrated algorithm ..."
Abstract

Cited by 39 (2 self)
 Add to MetaCart
Abstract. We provide a natural learning process in which the joint frequency of empirical play converges into the set of convex combinations of Nash equilibria. In this process, all players rationally choose their actions using a public prediction made by a deterministic, weakly calibrated algorithm. Furthermore, the public predictions used in any given round of play are frequently close to some Nash equilibrium of the game. 1
Probabilistic forecasts, calibration and sharpness
 Journal of the Royal Statistical Society Series B
, 2007
"... Summary. Probabilistic forecasts of continuous variables take the form of predictive densities or predictive cumulative distribution functions. We propose a diagnostic approach to the evaluation of predictive performance that is based on the paradigm of maximizing the sharpness of the predictive dis ..."
Abstract

Cited by 38 (15 self)
 Add to MetaCart
Summary. Probabilistic forecasts of continuous variables take the form of predictive densities or predictive cumulative distribution functions. We propose a diagnostic approach to the evaluation of predictive performance that is based on the paradigm of maximizing the sharpness of the predictive distributions subject to calibration. Calibration refers to the statistical consistency between the distributional forecasts and the observations and is a joint property of the predictions and the events that materialize. Sharpness refers to the concentration of the predictive distributions and is a property of the forecasts only. A simple theoretical framework allows us to distinguish between probabilistic calibration, exceedance calibration and marginal calibration. We propose and study tools for checking calibration and sharpness, among them the probability integral transform histogram, marginal calibration plots, the sharpness diagram and proper scoring rules. The diagnostic approach is illustrated by an assessment and ranking of probabilistic forecasts of wind speed at the Stateline wind energy centre in the US Pacific Northwest. In combination with crossvalidation or in the time series context, our proposal provides very general, nonparametric alternatives to the use of information criteria for model diagnostics and model selection.
Conditional Universal Consistency
, 1997
"... Each period, a player must choose an action without knowing the outcome that will be chosen by "Nature," according to an unknown and possibly historydependent stochastic rule. We discuss have a class of procedures that assign observations to categories, and prescribe a simple randomized variation o ..."
Abstract

Cited by 34 (0 self)
 Add to MetaCart
Each period, a player must choose an action without knowing the outcome that will be chosen by "Nature," according to an unknown and possibly historydependent stochastic rule. We discuss have a class of procedures that assign observations to categories, and prescribe a simple randomized variation of fictitious play within each category. These procedures are "conditionally consistent," in the sense of yielding almost as high a timeaverage payoff as could be obtained if the player chose knowing the conditional distributions of actions given categories. Moreover given any alternative procedure, there is a conditionally consistent procedure whose performance is no more than epsilon worse regardless of the discount factor. Cycles can persist if all players classify histories in the same way; however in an example, where players classify histories differently, the system converges to a Nash equilibrium. We also argue that in the long run the timeaverage of play should resemble a correlated equilibrium.
Calibrated Forecasting and Merging
, 1996
"... Consider a general finitestate stochastic process governed by an unknown objective probability distribution. Observing the system, a forecaster assigns subjective probabilities to future states. The resulting subjective forecast merges to the objective distribution if, with time, the forecasted pro ..."
Abstract

Cited by 22 (5 self)
 Add to MetaCart
Consider a general finitestate stochastic process governed by an unknown objective probability distribution. Observing the system, a forecaster assigns subjective probabilities to future states. The resulting subjective forecast merges to the objective distribution if, with time, the forecasted probabilities converge to the correct (but unknown) probabilities. The forecast is calibrated if observed longrun empirical distributions coincide with the forecasted probabilities. This paper links the unobserved reliability of forecasts to their observed empirical performance by demonstrating full equivalence between notions of merging and of calibration. It also indicates some implications of this equivalence for the literatures of forecasting and learning.
Any inspection is manipulable
 Econometrica
, 2001
"... Abstract. A forecaster provides a probabilistic prediction regarding the following day’s state of nature. To examine the forecaster, an inspector employs calibration tests that compare the average prediction and the empirical frequency of prespecified events. This paper shows that any mixed test ca ..."
Abstract

Cited by 17 (1 self)
 Add to MetaCart
Abstract. A forecaster provides a probabilistic prediction regarding the following day’s state of nature. To examine the forecaster, an inspector employs calibration tests that compare the average prediction and the empirical frequency of prespecified events. This paper shows that any mixed test can be manipulated in the sense that, independently of the state realizations, the difference between the average prediction and the past empirical frequency that corresponds to almost any test employed diminishes to zero. In other words, a forecaster has a prediction scheme that passes almost any test. In particular, a forecaster can pass all the tests in a countable set simultaneously. I am grateful to Rann Smorodinsky, Sylvain Sorin and two anonymous referees for their helpful
A geometric proof of calibration
 hal00773218, version 1  12
, 2013
"... We provide yet another proof of the existence of calibrated forecasters; it has two merits. First, it is valid for an arbitrary finite number of outcomes. Second, it is short and simple and it follows from a direct application of Blackwell’s approachability theorem to carefully chosen vectorvalued ..."
Abstract

Cited by 9 (6 self)
 Add to MetaCart
We provide yet another proof of the existence of calibrated forecasters; it has two merits. First, it is valid for an arbitrary finite number of outcomes. Second, it is short and simple and it follows from a direct application of Blackwell’s approachability theorem to carefully chosen vectorvalued payoff function and convex target set. Our proof captures the essence of existing proofs based on approachability (e.g., the proof by Foster [5] in case of binary outcomes) and highlights the intrinsic connection between approachability and calibration.
FirstOrder Bayesian Logic
, 2005
"... Uncertainty is a fundamental and irreducible aspect of our knowledge about the world. Until recently, classical firstorder logic has reigned as the de facto standard logical foundation for artificial intelligence. The lack of a builtin, semantically grounded capability for reasoning under uncertai ..."
Abstract

Cited by 8 (3 self)
 Add to MetaCart
Uncertainty is a fundamental and irreducible aspect of our knowledge about the world. Until recently, classical firstorder logic has reigned as the de facto standard logical foundation for artificial intelligence. The lack of a builtin, semantically grounded capability for reasoning under uncertainty renders classical firstorder logic inadequate for many important classes of problems. Generalpurpose languages are beginning to emerge for which the fundamental logical basis is probability. Increasingly expressive probabilistic languages demand a theoretical foundation that fully integrates classical firstorder logic and probability. In firstorder Bayesian logic (FOBL), probability distributions are defined over interpretations of classical firstorder axiom systems. Predicates and functions of a classical firstorder theory correspond to a random variables in the corresponding firstorder Bayesian theory. This is a natural correspondence, given that random variables are formalized in mathematical statistics as measurable functions on a probability space. A formal system called MultiEntity Bayesian Networks (MEBN) is presented for composing distributions on interpretations by instantiating and combining parameterized fragments of directed graphical models. A construction is given of a MEBN theory that assigns a nonzero
Consistency and Cautious Fictitious Play
, 1994
"... We study a variation of fictitious play, in which the probability of each action is an exponential function of that action's utility against the historical frequency of opponents' play. Regardless of the opponents' strategies, the utility received by an agent using this rule is nearly the best that ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
We study a variation of fictitious play, in which the probability of each action is an exponential function of that action's utility against the historical frequency of opponents' play. Regardless of the opponents' strategies, the utility received by an agent using this rule is nearly the best that could be achieved against the historical frequency. Such rules are approximately optimal in i.i.d. environments, and guarantee nearly the minmax regardless of opponents' behavior. Fictitious play shares these properties provided it switches "infrequently" between actions. We also study the long run outcomes when all players use consistent and cautious rules.