Results 1  10
of
38
Multivariate Density Forecast Evaluation and Calibration
 in Financial Risk Management: HighFrequency Returns on Foreign Exchange,” Review of Economics and Statistics
, 1999
"... educational and research purposes, so long as it is not altered, this copyright notice is reproduced with it, and it is not sold for profit. Abstract: We provide a framework for evaluating and improving multivariate density forecasts. Among other things, the multivariate framework lets us evaluate t ..."
Abstract

Cited by 72 (15 self)
 Add to MetaCart
educational and research purposes, so long as it is not altered, this copyright notice is reproduced with it, and it is not sold for profit. Abstract: We provide a framework for evaluating and improving multivariate density forecasts. Among other things, the multivariate framework lets us evaluate the adequacy of density forecasts involving crossvariable interactions, such as timevarying conditional correlations. We also provide conditions under which a technique of density forecast “calibration ” can be used to improve deficient density forecasts. Finally, motivated by recent advances in financial risk management, we provide a detailed application to multivariate highfrequency exchange rate density forecasts.
Density Forecasting: A Survey
 Journal of Forecasting
, 2000
"... A density forecast of the realization of a random variable at some future time is an estimate of the probability distribution of the possible future values of that variable. This chapter presents a selective survey of applications of density forecasting in macroeconomics and finance, and discusses s ..."
Abstract

Cited by 65 (9 self)
 Add to MetaCart
A density forecast of the realization of a random variable at some future time is an estimate of the probability distribution of the possible future values of that variable. This chapter presents a selective survey of applications of density forecasting in macroeconomics and finance, and discusses some issues concerning the production, presentation, and evaluation of density forecasts. This chapter first appeared as an article with the same title in Journal of Forecasting, 19 (2000), 235254. The helpful comments and suggestions of Frank Diebold, Stewart Hodges and two anonymous referees are gratefully acknowledged. Subsequent editorial changes have been made following suggestions from the editors of this volume. Responsibility for errors remains with the authors. 2 1. INTRODUCTION A density forecast of the realization of a random variable at some future time is an estimate of the probability distribution of the possible future values of that variable. It thus provides a complet...
Probabilistic forecasts, calibration and sharpness
 Journal of the Royal Statistical Society Series B
, 2007
"... Summary. Probabilistic forecasts of continuous variables take the form of predictive densities or predictive cumulative distribution functions. We propose a diagnostic approach to the evaluation of predictive performance that is based on the paradigm of maximizing the sharpness of the predictive dis ..."
Abstract

Cited by 38 (15 self)
 Add to MetaCart
Summary. Probabilistic forecasts of continuous variables take the form of predictive densities or predictive cumulative distribution functions. We propose a diagnostic approach to the evaluation of predictive performance that is based on the paradigm of maximizing the sharpness of the predictive distributions subject to calibration. Calibration refers to the statistical consistency between the distributional forecasts and the observations and is a joint property of the predictions and the events that materialize. Sharpness refers to the concentration of the predictive distributions and is a property of the forecasts only. A simple theoretical framework allows us to distinguish between probabilistic calibration, exceedance calibration and marginal calibration. We propose and study tools for checking calibration and sharpness, among them the probability integral transform histogram, marginal calibration plots, the sharpness diagram and proper scoring rules. The diagnostic approach is illustrated by an assessment and ranking of probabilistic forecasts of wind speed at the Stateline wind energy centre in the US Pacific Northwest. In combination with crossvalidation or in the time series context, our proposal provides very general, nonparametric alternatives to the use of information criteria for model diagnostics and model selection.
Evaluating, comparing and combining density forecasts using the KLIC with an application to the Bank of England and NIESR “fan” charts of inflation
, 2005
"... ..."
Optimal combination of density forecasts
 NATIONAL INSTITUTE OF ECONOMIC AND SOCIAL RESEARCH DISCUSSION PAPER NO
, 2005
"... This paper brings together two important but hitherto largely unrelated areas of the forecasting literature, density forecasting and forecast combination. It proposes a simple datadriven approach to direct combination of density forecasts using optimal weights. These optimal weights are those weigh ..."
Abstract

Cited by 23 (9 self)
 Add to MetaCart
This paper brings together two important but hitherto largely unrelated areas of the forecasting literature, density forecasting and forecast combination. It proposes a simple datadriven approach to direct combination of density forecasts using optimal weights. These optimal weights are those weights that minimize the ‘distance’, as measured by the KullbackLeibler information criterion, between the forecasted and true but unknown density. We explain how this minimization both can and should be achieved. Comparisons with the optimal combination of point forecasts are made. An application to simple timeseries density forecasts and two widely used published density forecasts for U.K. inflation, namely the Bank of England and NIESR “fan” charts, illustrates that combination can but need not always help.
Economic forecasting: some lessons from recent research
, 2002
"... This paper describes some recent advances and contributions to our understanding of economic forecasting. The framework we develop helps explain the findings of forecasting competitions and the prevalence of forecast failure. It constitutes a general theoretical background against which recent resul ..."
Abstract

Cited by 19 (2 self)
 Add to MetaCart
This paper describes some recent advances and contributions to our understanding of economic forecasting. The framework we develop helps explain the findings of forecasting competitions and the prevalence of forecast failure. It constitutes a general theoretical background against which recent results can be judged. We compare this framework to a previous formulation, which was silent on the very issues of most concern to the forecaster. We describe a number of aspects which it illuminates, and draw out the implications for model selection. Finally, we discuss the areas where research remains needed to clarify empirical findings which lack theoretical explanations.
Evaluating the Survey of Professional Forecasters probability distributions of expected inflation based on derived probability forecasts
, 2005
"... Regressionbased tests of forecast probabilities of particular events of interest are constructed. The event forecast probabilities are derived from the SPF density forecasts of expected inflation and output growth. Tests of the event probabilities supplement statisticallybased assessments of the f ..."
Abstract

Cited by 16 (3 self)
 Add to MetaCart
Regressionbased tests of forecast probabilities of particular events of interest are constructed. The event forecast probabilities are derived from the SPF density forecasts of expected inflation and output growth. Tests of the event probabilities supplement statisticallybased assessments of the forecast densities using the probability integral transform approach. The regressionbased tests assess whether the forecast probabilities of particular events are equal to the true probabilities, and whether any systematic divergences between the two are related to variables in the agents ’ information set at the time the forecasts were made. Forecast encompassing tests are also used to assess the quality of the event probability forecasts.
Comparing density forecast models
 University of California, Riverside
, 2007
"... In this paper we discuss how to compare various (possibly misspecified) density forecast models using the KullbackLeibler Information Criterion (KLIC) of a candidate density forecast model with respect to thetruedensity. TheKLICdifferential between a pair of competing models is the (predictive) lo ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
In this paper we discuss how to compare various (possibly misspecified) density forecast models using the KullbackLeibler Information Criterion (KLIC) of a candidate density forecast model with respect to thetruedensity. TheKLICdifferential between a pair of competing models is the (predictive) loglikelihood ratio (LR) between the two models. Even though the true density is unknown, using the LR statistic amounts to comparing models with the KLIC as a loss function and thus enables us to assess which density forecast model can approximate the true density more closely. We also discuss how this KLIC is related to the KLIC based on the probability integral transform (PIT) in the framework of Diebold et al. (1998). While they are asymptotically equivalent, the PITbased KLIC is best suited for evaluating the adequacy of each density forecast model and the original KLIC is best suited for comparing competing models. In an empirical study with the S&P500 and NASDAQ daily return series, we find strong evidence for rejecting the NormalGARCH benchmark model, in favor of the models that can capture skewness in the conditional distribution and asymmetry and longmemory in the conditional variance.
A Test for Density Forecast Comparison with Applications to Risk Management
, 2004
"... In this paper we propose a testing procedure for comparing the predictive abilities of possibly misspecified density forecast models. We use the minimum KullbackLeibler Information Criterion (KLIC) divergence measure to define the distance between the candidate density forecast model and the true m ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
In this paper we propose a testing procedure for comparing the predictive abilities of possibly misspecified density forecast models. We use the minimum KullbackLeibler Information Criterion (KLIC) divergence measure to define the distance between the candidate density forecast model and the true model. We use the fact that the inversenormal transform of the probability integral transforms (PIT) should be IID standard normal as discussed in Berkowitz (2001). To compare the performance of density forecast models in the tails, we use the censored likelihood functions to compute the tail minimum KLIC. The reality check test of White (2000) is then constructed using our distance measure as a loss function. To highlight the merits of our approach, we use the daily S&P500 and NASDAQ return series to conduct an empirical density forecast comparison exercise. A large set of distributions, including some recently proposed flexible distributions to accommodate higher moments, and the ARCHfamily volatility specifications are studied. Our empirical findings lend further support of fattailedness and skewness of return distributions. In addition, the choice of conditional distribution specification appears to be a much more dominant factor in determining the quality of density forecasts than the choice of volatility specification.