It is obvious that forecasts are of great importance and widely used in economics and finance. Quite simply, good forecasts lead to good decisions. The importance of forecast evaluation and combination techniques follows immediately-- forecast users naturally have a keen interest in monitoring and improving forecast performance. More generally, forecast evaluation figures prominently in many questions in empirical economics and finance, such as: Are expectations rational? (e.g., Keane and Runkle, 1990; Bonham and Cohen, 1995) Are financial markets efficient? (e.g., Fama, 1970, 1991) Do macroeconomic shocks cause agents to revise their forecasts at all horizons, or just at short- and medium-term horizons? (e.g., Campbell and Mankiw, 1987; Cochrane, 1988) Are observed asset returns "too volatile"? (e.g., Shiller, 1979; LeRoy and Porter, 1981) Are asset returns forecastable over long horizons? (e.g., Fama and French, 1988; Mark, 1995)

We argue that the current framework for predictive ability testing (e.g.,West, 1996) is not necessarily useful for real-time forecast selection, i.e., for assessing which of two competing forecasting methods will perform better in the future. We propose an alternative framework for out-of-sample comparison of predictive ability which delivers more practically relevant conclusions. Our approach is based on inference about conditional expectations of forecasts and forecast errors rather than the unconditional expectations that are the focus of the existing literature. We capture important determinants of forecast performance that are neglected in the existing literature by evaluating what we call the forecasting method (the model and the parameter estimation procedure), rather than just the forecasting model. Compared to previous approaches, our tests are valid under more general data assumptions (heterogeneity rather than stationarity) and estimation methods, and they can handle comparison of both nested and non-nested models, which is not currently possible. To illustrate the usefulness of the proposed tests, we compare the forecast performance of three leading parameter-reduction methods for macroeconomic forecasting using a large number of predictors: a sequential model selection approach,

Statistical loss functions that generally lack economic content are commonly used for evaluating financial volatility forecasts. In this paper, an evaluation framework based on loss functions tailored to a user’s economic interests is proposed. According to these interests, the user specifies the economic events to be forecast, the criterion with which to evaluate these forecasts, and the subsets of the forecasts of particular interest. The volatility forecasts from a model are then transformed into probability forecasts of the relevant events and evaluated using the specified criteria (i.e., a probability scoring rule and calibration tests). An empirical example using exchange rate data illustrates the framework and confirms that the choice of loss function directly affects the forecast evaluation results.

Abstract not found

In many applications it is necessary to use a simple and therefore highly misspecified econometric model as the basis for decision-making. We propose an approach to developing a possibly misspecified econometric model that will be used as the beliefs of an objective expected utility maximiser. A discrepancy between model and ‘truth ’ is introduced that is interpretable as a measure of the model’s value for this decision-maker. Our decision-based approach utilises this discrepancy in estimation, selection, inference and evaluation of parametric or semiparametric models. The methods proposed nest quasilikelihood methods as a special case that arises when model value is measured by the Kullback-Leibler information discrepancy and also provide an econometric approach for developing parametric decision rules (e.g. technical trading rules) with desirable properties. The approach is illustrated and applied in the context of a CARA investor’s decision problem for which analytical, simulation and empirical results suggest it is very effective.

and participants at the 2001 Cambridge-Maastricht Symposium for Real Estate Finance and Economics for helpful comments. Remaining errors are, of course, the responsibility of the authors. Previous research has shown that the returns on individual properties and listed property securities are skewed (Lizieri and Ward 2001, Young and Graff 1995 and Liu et al. 1992). This claim is investigated in the context of listed UK property companies and US REITs. In particular, the shape of the conditional distribution of total monthly returns is examined for a group of 20 UK companies and 20 REITS listed continuously since 1970 and 1977, respectively. Also investigated is the claim of Young and Graff that the skewness found in property returns varies over time. Using the model of Hansen (1994) it is found that while a large portion of property security returns in the sample do exhibit skewness in the conditional distribution only in a few instances is there evidence of time variation in the skewness parameter. When time varying skewness is found there is little evidence to suggest it is associated with the economic cycle. The link between time varying skewness models and downside risk measures is also discussed and estimates of conditional downside risk are calculated for those companies exhibiting the time varying skewness property.

this document, while Reuters will control the production and distribution of the RiskMetrics data sets. . Expanded sections on methodology outline enhanced analytical solutions for dealing with nonlinear options risks and introduce methods on how to account for non-normal distributions. . Enclosed diskette contains many examples used in this document. It allows readers to experiment with our risk measurement techniques. . All publications and daily data sets are available free of charge on J.P. Morgan's Web page on the Internet at http://www.jpmorgan.com/RiskManagement/RiskMetrics/RiskMetrics.html. This page is accessible directly or through third party services such as CompuServe, America Online , or Prodigy.

In situations where a sequence of forecasts is observed, a common strategy is to examine ‘rationality ’ conditional on a given loss function. We examine this from a different perspective- supposing that we have a family of loss functions indexed by unknown shape parameters, then given the forecasts can we back out the loss function parameters consistent with the forecasts being rational even when we do not observe the underlying forecasting model? We establish identification of the parameters of a general class of loss functions that nest popular loss functions as special cases and provide estimation methods and asymptotic distributional results for these parameters. The methods are applied in an empirical analysis of IMF and OECD forecasts of budget deficits for the G7 countries. We find that allowing for asymmetric loss can significantly change the outcome of empirical tests of forecast rationality. ∗We thank two anonymous referees, the editor, Hidehiko Ichimura, and seminar participants at UCLA,

• “Decision-based methods for forecast evaluation”, (with M. Hashem Pesaran) in Companion to Economic Forecasting, M.P. Clements and D.F. Hendry (Eds), 2001,

This paper focuses on the performance of various GARCH models in terms of their ability of delivering volatility forecasts for stock return data. Volatility forecasts obtained from a variety of mean and variance specifications in GARCH models are compared to a proxy of actual volatility calculated using daily data. In-sample tests suggest that a regression of volatility estimates on actual volatility produces R2soflessthan8%. An interesting by-product is evidence of significantly negative relation between unexpected volatility and stock returns. Finally, out-of-sample tests indicate that a simpler ARMA specification performs better than a GARCH-M model.