## Evaluating Density Forecasts: Is Sharpness Needed? (2008)

### BibTeX

@MISC{Mitchell08evaluatingdensity,

author = {James Mitchell and Kenneth F. Wallis},

title = {Evaluating Density Forecasts: Is Sharpness Needed?},

year = {2008}

}

### OpenURL

### Abstract

Summary. In a recent article Gneiting, Balabdaoui and Raftery (Journal of the Royal Statistical Society B, 2007) propose the criterion of sharpness for the evaluation of predictive distributions or density forecasts. They motivate their proposal by an example in which standard evaluation procedures based on probability integral transforms cannot distinguish between the ideal forecast and several competing forecasts. In this paper we show that their example has some unrealistic features which make it an insecure foundation for their argument that existing calibration procedures are inadequate in practice. We present an alternative, more realistic example in which relevant statistical methods provide the required discrimination between competing forecasts, and argue that there is no need for a subsidiary criterion of sharpness.

### Citations

302 | Evaluating density forecasts with application to financial risk management - Diebold, Gunther, et al. - 1998 |

248 | Finite Mixture Distributions - Everitt, Hand - 1981 |

188 |
Statistical theory. The prequential approach
- Dawid
- 1984
(Show Context)
Citation Context ...ly; this ‘has an obvious analogy with the Likelihood Principle, in asserting the irrelevance of hypothetical forecasts that might have been issued in circumstances that did not, in fact, come about’ (=-=Dawid, 1984-=-, p.281). A standard approach is to calculate the probability integral transform values of the outcomes in the forecast distributions, and assessment rests on ‘the question of whether [such] a sequenc... |

170 | Rational decisions - Good - 1952 |

163 |
Testing Density Forecasts with Applications to Risk Management
- Berkowitz
- 2001
(Show Context)
Citation Context ...ess-of-fit can be based on the PIT values −1 their inverse normal transformation, z ( ) t =Φ t p t or on the values given by p , where Φ() ⋅ is the standard normal distribution function (Smith, 1985; =-=Berkowitz, 2001-=-). If p t is iidU(0,1), then zt is iidN(0,1). 10The advantages of this second transformation are that there are more tests available for normality, it is easier to test autocorrelation under normalit... |

79 | Density forecasting: a survey - Tay, Wallis |

78 | An omnibus test for univariate and multivariate normality. Working paper, Nuffield
- Doornik, Hansen
- 1994
(Show Context)
Citation Context ..., which could be y calculated directly. t t We consider the classical Kolmogorov-Smirnov (KS) and Anderson-Darling (AD) tests for uniformity, together with the Doornik-Hansen (DH) test for normality (=-=Doornik and Hansen, 1994-=-). These are all based on random sampling assumptions, and there are no general results about their performance under autocorrelation. Table 2 reports the rejection percentages for each of these goodn... |

64 | 2001: Interpretation of rank histograms for verifying ensemble forecasts - Hamill |

47 | Probabilistic forecasts, calibration and sharpness - Gneiting, Balabdaoui, et al. - 2005 |

39 | Evaluating, comparing and combining density forecasts using the KLIC with an application to the Bank of England and NIESR ‘fan’ charts of inflation - Mitchell, Hall - 2005 |

32 | An evaluation of tests of distributional forecasts - Noceti, Smith, et al. - 2003 |

21 |
Diagnostic checks of non-standard time series models
- Smith
- 1985
(Show Context)
Citation Context ...ests of goodness-of-fit can be based on the PIT values −1 their inverse normal transformation, z ( ) t =Φ t p t or on the values given by p , where Φ() ⋅ is the standard normal distribution function (=-=Smith, 1985-=-; Berkowitz, 2001). If p t is iidU(0,1), then zt is iidN(0,1). 10The advantages of this second transformation are that there are more tests available for normality, it is easier to test autocorrelati... |

16 | Comparing density forecast models - Bao, Lee, et al. - 2007 |

15 | Editorial: Probabilistic forecasting - Gneiting |

5 | Forecasting white noise - Granger - 1983 |

1 | First circulated as ‘A test for density forecast comparison with applications to risk management - Forecast - 2004 |