Abstract—A common problem that arises in adaptive filtering, autoregressive modeling, or linear prediction is the selection of an appropriate order for the underlying linear parametric model. We address this problem for linear prediction, but instead of fixing a specific model order, we develop a sequential prediction algorithm whose sequentially accumulated average squared prediction error for any bounded individual sequence is as good as the performance attainable by the best sequential linear predictor of order less than some w. This predictor is found by transforming linear prediction into a problem analogous to the sequential probability assignment problem from universal coding theory. The resulting universal predictor uses essentially a performance-weighted average of all predictors for model orders less than w. Efficient lattice filters are used to generate the predictions of all the models recursively, resulting in a complexity of the universal algorithm that is no larger than that of the largest model order. Examples of prediction performance are provided for autoregressive and speech data as well as an example of adaptive data equalization. Index Terms—Adaptive filters, Bayes procedures, learning systems, least squares methods, model order, prediction methods,

nding board both for the general struggles of graduate school as well as for technical ideas. Our families also supplied large doses of encouragement, enthusiasm, and understanding. My mother was a particular inspiration, often providing reminders of the many "words of wisdom" I iii had given her during her graduate school days. Finally, I would like to thank my God, who alone provides the talents, the opportunity, and the strength for such an undertaking. "Unless the Lord builds the house, they labor in vain who build it." Soli Deo Gloria. iv Contents Acknowledgments ii List of Tables viii List of Figures ix Summary xii 1 Introduction 1 1.1 Statement of the Problem : : : : : : : : : : : : : : : : : : : : : : : : 1 1.2 Scope of the Thesis : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5 2 Background 8 2.1 Image Formation : : : : : : : : : : : : : : : : : : : : : : : : : : : : :<F

The predictive capability of a modification of Rissanen’s accumulated prediction error (APE) criterion, APEδn, is investigated in infinite-order autoregressive (AR(∞)) models. Instead of accumulating squares of sequential prediction errors from the beginning, APEδn is obtained by summing these squared errors from stage nδn, where n is the sample size and 1/n ≤ δn ≤ 1 −(1/n) may depend on n. Under certain regularity conditions, an asymptotic expression is derived for the mean-squared prediction error (MSPE) of an AR predictor with order determined by APEδn. This expression shows that the prediction performance of APEδn can vary dramatically depending on the choice of δn. Another interesting finding is that when δn approaches 1 at a certain rate, APEδn can achieve asymptotic efficiency in most practical situations. An asymptotic equivalence between APEδn and an information criterion with a suitable penalty term is also established from the MSPE point of view. This offers new perspectives for understanding the information and prediction-based model selection criteria. Finally, we provide the first asymptotic efficiency result for the case when the underlying AR(∞) model is allowed to degenerate to a finite autoregression.

dedicated to

We investigate the predictive ability of the accumulated prediction error (APE) of Rissanen in an infinite-order autoregressive (AR(#)) model. Since there are infinitely many parameters in the model, all finite-order AR models are misspecified. We first show that APE is asymptotically equivalent to Bayesian information criterion (BIC) and is not asymptotically e#cient in the misspecified case. To rectify this di#culty, a modification of APE, APE # , is proposed. Instead of accumulating squares of sequential prediction errors from the beginning, APE # is obtained by accumulating squares of sequential prediction errors from stage n#, where n is the sample size and 0 < # < 1 may depend on n. Under certain regularity conditions, we show that APE # is asymptotically e#cient in the sense that the mean-squared prediction error (MSPE) of the AR model with order selected by APE # can ultimately achieve the best compromise between model complexity and the goodness of fit. Based on this result, we further show that APE # is asymptotically equivalent to Akaike's information criterion (AIC) in an AR(#) model. This is a somewhat interesting discovery because the proposed modification totally changes the nature of APE from a BIC-like criterion to an AIC-like criterion. An extensive simulation study is given to illustrate this theoretical result. We surprisingly found from the simulation results that APE # has uniformly better finite-sample performance than both AIC and BIC. Finally, a suitable choice of # is suggested for n between 100 and 500.

Abstract not found

Autoregressive (AR) modeling by linear prediction (LP) provides the basis of a wide variety of signal processing and communication systems including parametric spectral estimation and system identification. Perhaps the greatest success of linear prediction techniques is to be found in speech analysis and audio coding. In this paper, we first reviewed the general frameworks of predictive signal modeling and investigated various prediction filter structures including the modified linear predictor. We then empirically compared the compression performamce of these prediction filters by applying to the lossless audio compression system. We also applied different filter orders and block lengths for each filter to explore their influence on the compression ratio....

Abstract: In this paper the stochastic complexity criterion is applied to estimation of the order in AR and ARMA models. The power of the criterion for short strings is illustrated by simulations. It requires an integral of the square root of Fisher information, which is done by Monte Carlo technique. The stochastic complexity, which is the negative logarithm of the Normalized Maximum Likelihood universal density function, is given. Also, exact asymptotic formulas for the Fisher information matrix are derived. 1.