We derive a Minimum Message Length (MML) estimator for stationary and nonstationary autoregressive models using the Wallace and Freeman (1987) approximation. The MML estimator’s model selection performance is empirically compared with AIC, AICc, BIC and HQ in a Monte Carlo experiment by uniformly sampling from the autoregressive stationarity region. Generally applicable, uniform priors are used on the coefficients, model order and log σ 2 for the MML estimator. The experimental results show the MML estimator to have the best overall average mean squared prediction error and best ability to choose the true model order.

Statistical models (e.g., ARIMA models) have been commonly used in time series data analysis and forecasting. Typically one model is selected based on a selection criterion (e.g., AIC), hypothesis testing, and/or graphical inspections. The selected model is then used to forecast future values. However, model selection is often unstable and may cause an unnecessarily high variability in the final estimation/prediction. In this work, we propose the use of an algorithm AFTER to convexly combine the models for a better performance of prediction. The weights are sequentially updated after each additional observation. Simulations and real data examples are used to compare performance of our approach with model selection methods. The results show advantage of combining by AFTER over selection in term of forecasting accuracy at several settings.

The Akaike information criterion, AIC, is a widely known and extensively used tool for statistical model selection. AIC serves as an asymptotically unbiased estimator of a variant of Kullback's directed divergence between the true model and a fitted approximating model. The directed divergence is an asymmetric measure of separation between two statistical models, meaning that an alternate directed divergence may be obtained by reversing the roles of the two models in the definition of the measure. The sum of the two directed divergences is Kullback's symmetric divergence. Since the symmetric divergence combines the information in two related though distinct measures, it functions as a gauge of model disparity which is arguably more sensitive than either of its individual components. With this motivation, we propose a model selection criterion which serves as an asymptotically unbiased estimator of a variant of the symmetric divergence between the true model and a fitted approximating model. We examine the performance of the criterion relative to other well-known criteria in a simulation study. Keywords: AIC, Akaike information criterion, I-divergence, J-divergence, Kullback-Leibler information, relative entropy. Correspondence: Joseph E. Cavanaugh, Department of Statistics, 222 Math Sciences Bldg., University of Missouri, Columbia, MO 65211. y This research was supported by NSF grant DMS--9704436. 1.

The recent controversy over model selection in the context of ‘growth regressions ’ has led to some remarkably numerous ‘estimation ’ strategies, including 4 million regressions by Sala-i-Martin (1997b). Only one regression is really needed, namely the general unrestricted model, appropriately reduced to a parsimonious encompassing congruent representation. Such an outcome was achieved in one run on PcGets, within 15 minutes of receiving from Professor Ley the data set in Fernández et al (2001). We reproduce that equation, and corroborate the findings in Hoover and Perez (2004), who also adopt an automatic general-to-simple approach.

In this paper we consider a three-step procedure for identification of timeseries, based on covariance extension and modelreduction, and we present a complete analysis of its statistical convergence properties. A partial covariance sequence is estimated from statistical data. Then a high-order maximum-entropy model is determined, which is finally approximated by a lower-order model by stochastically balanced model reduction. Such procedures have been studied before, in various combinations, but an overall convergence analysis comprising all three steps has been lacking. Supposing the data is generated from a true finitedimensional system which is minimumphase, it is shown that the transfer function of the estimated system tends in H # tothe true transfer function as the data length tends to infinity, if the covariance extension and the model reduction is done properly. The proposed identification procedure, and some variations ofit, are evaluated by simulations. 1.

this paper were collected in forested regions of northern Hokkaido (Fig. 1; 41

: In some applications, the mean or median response is linearly related to some variables but the relation to additional variables are not easily parameterized. Partly linear models arise naturally in such circumstances. Suppose that a random sample f(T i ; X i ; Y i ); i = 1; 2; \Delta \Delta \Delta ; ng is modeled by Y i = X T i fi 0 + g 0 (T i ) + error i , where Y i is a real-valued response, X i 2 R p and T i ranges over a unit square, and g 0 is an unknown function with a certain degree of smoothness. We make use of bivariate tensor-product B-splines as an approximation of the function g 0 and consider M-type regression splines by minimization of P n i=1 ae(Y i \Gamma X T i fi \Gamma g n (T i )) for some convex function ae. Mean, median and quantile regressions are included in this class. We show under appropriate conditions that the parameter estimate of fi achieves its information bound asymptotically and the function estimate of g 0 attains the optimal rate of convergen...

We investigate the task of performance prediction for language models belonging to the exponential family. First, we attempt to empirically discover a formula for predicting test set cross-entropy for n-gram language models. We build models over varying domains, data set sizes, and n-gram orders, and perform linear regression to see whether we can model test set performance as a simple function of training set performance and various model statistics. Remarkably, we find a simple relationship that predicts test set performance with a correlation of 0.9997. We analyze why this relationship holds and show that it holds for other exponential language models as well, including class-based models and minimum discrimination information models. Finally, we discuss how this relationship can be applied to improve language model performance. 1

We derive and analyze a suboptimal algorithm for estimating the unknown parameters of an exponential polynomial signal (EPS) model. The EPS model represents a signal as an exponent raised to a complex polynomial of time. That is, the phase and the logarithm of the amplitude are approximated as finite-order polynomials of time. Index Terms---Chirps, maximum likelihood, parameter estimation. I. INTRODUCTION In this correspondence, we examine the estimation of complex signals when both the amplitude and the phase of these signals vary with time. The method is based on modeling the phase of the signal and the logarithm of its time-varying amplitude as a finite-order polynomial of time. Thus, the signal can be modeled as an exponent raised to a complex polynomial. The real parts of the polynomial coefficients specify the envelope of the signal, whereas the imaginary parts specify its phase. We refer to such a signal as an exponential polynomial signal (EPS). If the data are real rather t...

The generation of rainfall and other climate data needs a range of models depending on the time and spatial scales involved. Most of the models used previously do not take into account year to year variations in the model parameters. Long periods of wet and dry years were observed in the past but were not taken into account. Recently, Thyer and Kuczera (1999) developed a hidden state Markov model to account for the wet and dry spells explicitly in annual rainfall. This review looks firstly at traditional time series models and then at the more complex models which take account of the pseudo-cycles in the data. Monthly rainfall data have been generated successfully by using the method of fragments. The main criticism of this approach is the repetitions of the same yearly pattern when only a limited number of years of historical data are available. This deficiency has been overcome by using synthetic fragments but this brings an additional problem of generating the right number of months with zero rainfall. Disaggregation schemes are effective in obtaining monthly data but the main problem is the large number of parameters to be estimated when dealing with many sites. Several simplifications have been proposed to overcome this problem. Models for generating daily rainfall are well developed. The transition probability matrix method preserves most of the characteristics of daily, monthly and annual characteristics and is shown to be the best performing model. The two-part model has been shown by many researchers to perform well across a range of climates at the daily level but has not been tested adequately at monthly or annual levels. A shortcoming of the existing models is the consistent underestimation of the variances of the simulated monthly and annual totals. As an alternative, conditioning model parameters on monthly amounts or perturbing the model parameters with the Southern Oscillation Index