This paper reviews the principle of Minimum Description Length (MDL) for problems of model selection. By viewing statistical modeling as a means of generating descriptions of observed data, the MDL framework discriminates between competing models based on the complexity of each description. This approach began with Kolmogorov's theory of algorithmic complexity, matured in the literature on information theory, and has recently received renewed interest within the statistics community. In the pages that follow, we review both the practical as well as the theoretical aspects of MDL as a tool for model selection, emphasizing the rich connections between information theory and statistics. At the boundary between these two disciplines, we find many interesting interpretations of popular frequentist and Bayesian procedures. As we will see, MDL provides an objective umbrella under which rather disparate approaches to statistical modeling can co-exist and be compared. We illustrate th...

Abstract not found

this paper I discuss a Bayesian approach to solving this problem that has long been available in principle but is only now becoming routinely feasible, by virtue of recent computational advances, and examine its implementation in examples that involve forecasting the price of oil and estimating the chance of catastrophic failure of the U.S. Space Shuttle.

by Rebecka Jenny Jornsten Doctor of Philosophy in Statistics University of California, Berkeley Professor Bin Yu, Chair This thesis consists of three parts. Even though each part is self-contained, a common theme runs through all of them: data compression and its implications for statistical inference. In particular, we consider the following three questions. How can we quantify the effect of compression on statistical inference? How should a compression scheme be designed such that the effect of compression on inference is minimal? How can the Minimum Description Length (MDL) principle be used for model selection with an extraordinary number of dependent predictors? In this thesis, we attempt to answer these three questions in a general setting, and with a specific application in the compression and analysis of microarray images.

This paper considers the problem of estimating the dispersion parameter in a Gaussian model which is intermediate between a model where the mean parameter is fully known (fixed) and a model where the mean parameter is completely unknown. One of the goals is to understand the implications of the two-step process of first selecting a model among a finite number of sub-models, and then estimating a parameter of interest after the model selection, but using the same sample data. The estimators are classified into global, two-step, and weighted-type estimators. While the global-type estimators ignore the model space structure, the two-step estimators explore the structure adaptively and can be related to pre-test estimators, and the weighted estimators are motivated by the Bayesian approach. Their performances are compared theoretically and through simulations using their risk functions based on quadratic loss function. It is shown that in the variance estimation problem efficiency gains arise by exploiting the sub-model structure through the use of two-step and weighted estimators, especially when the number of competing sub-models is few; but that this advantage may deteriorate or be lost altogether for some two-step estimators as the number of sub-models increases or as the distance

Summary. For a continuous and diagonally symmetric multivariate distributian, incorporating the idea of preliminary test estimators, a variant form of the James-Stein type estimation rule is used to formulate some admissible R-estirnators of location. Asymptotic admissibility (and inadmissibility) results pertaining to the proposed and classical R-estimators are studied.