Results 1 
8 of
8
Model Selection and the Principle of Minimum Description Length
 Journal of the American Statistical Association
, 1998
"... This paper reviews the principle of Minimum Description Length (MDL) for problems of model selection. By viewing statistical modeling as a means of generating descriptions of observed data, the MDL framework discriminates between competing models based on the complexity of each description. This ..."
Abstract

Cited by 145 (5 self)
 Add to MetaCart
This paper reviews the principle of Minimum Description Length (MDL) for problems of model selection. By viewing statistical modeling as a means of generating descriptions of observed data, the MDL framework discriminates between competing models based on the complexity of each description. This approach began with Kolmogorov's theory of algorithmic complexity, matured in the literature on information theory, and has recently received renewed interest within the statistics community. In the pages that follow, we review both the practical as well as the theoretical aspects of MDL as a tool for model selection, emphasizing the rich connections between information theory and statistics. At the boundary between these two disciplines, we find many interesting interpretations of popular frequentist and Bayesian procedures. As we will see, MDL provides an objective umbrella under which rather disparate approaches to statistical modeling can coexist and be compared. We illustrate th...
Assessment and Propagation of Model Uncertainty
, 1995
"... this paper I discuss a Bayesian approach to solving this problem that has long been available in principle but is only now becoming routinely feasible, by virtue of recent computational advances, and examine its implementation in examples that involve forecasting the price of oil and estimating the ..."
Abstract

Cited by 108 (0 self)
 Add to MetaCart
this paper I discuss a Bayesian approach to solving this problem that has long been available in principle but is only now becoming routinely feasible, by virtue of recent computational advances, and examine its implementation in examples that involve forecasting the price of oil and estimating the chance of catastrophic failure of the U.S. Space Shuttle.
Data Compression and Its Statistical Implications, with an Application to the Analysis of Microarray Images
, 2001
"... by Rebecka Jenny Jornsten Doctor of Philosophy in Statistics University of California, Berkeley Professor Bin Yu, Chair This thesis consists of three parts. Even though each part is selfcontained, a common theme runs through all of them: data compression and its implications for statistical in ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
by Rebecka Jenny Jornsten Doctor of Philosophy in Statistics University of California, Berkeley Professor Bin Yu, Chair This thesis consists of three parts. Even though each part is selfcontained, a common theme runs through all of them: data compression and its implications for statistical inference. In particular, we consider the following three questions. How can we quantify the effect of compression on statistical inference? How should a compression scheme be designed such that the effect of compression on inference is minimal? How can the Minimum Description Length (MDL) principle be used for model selection with an extraordinary number of dependent predictors? In this thesis, we attempt to answer these three questions in a general setting, and with a specific application in the compression and analysis of microarray images.
Variance Estimation in a Model with Gaussian SubModels
, 2003
"... This paper considers the problem of estimating the dispersion parameter in a Gaussian model which is intermediate between a model where the mean parameter is fully known (fixed) and a model where the mean parameter is completely unknown. One of the goals is to understand the implications of the two ..."
Abstract
 Add to MetaCart
This paper considers the problem of estimating the dispersion parameter in a Gaussian model which is intermediate between a model where the mean parameter is fully known (fixed) and a model where the mean parameter is completely unknown. One of the goals is to understand the implications of the twostep process of first selecting a model among a finite number of submodels, and then estimating a parameter of interest after the model selection, but using the same sample data. The estimators are classified into global, twostep, and weightedtype estimators. While the globaltype estimators ignore the model space structure, the twostep estimators explore the structure adaptively and can be related to pretest estimators, and the weighted estimators are motivated by the Bayesian approach. Their performances are compared theoretically and through simulations using their risk functions based on quadratic loss function. It is shown that in the variance estimation problem efficiency gains arise by exploiting the submodel structure through the use of twostep and weighted estimators, especially when the number of competing submodels is few; but that this advantage may deteriorate or be lost altogether for some twostep estimators as the number of submodels increases or as the distance
OF MULTIVARIATE LOCATION' by
"... Summary. For a continuous and diagonally symmetric multivariate distributian, incorporating the idea of preliminary test estimators, a variant form of the JamesStein type estimation rule is used to formulate some admissible Restirnators of location. Asymptotic admissibility (and inadmissibility) r ..."
Abstract
 Add to MetaCart
Summary. For a continuous and diagonally symmetric multivariate distributian, incorporating the idea of preliminary test estimators, a variant form of the JamesStein type estimation rule is used to formulate some admissible Restirnators of location. Asymptotic admissibility (and inadmissibility) results pertaining to the proposed and classical Restimators are studied.
ARTICLE IN PRESS Journal of Statistical Planning and Inference ( ) – www.elsevier.com/locate/jspi A pretest like estimator dominating the leastsquares method �
, 2007
"... We develop a pretest type estimator of a deterministic parameter vector β in a linear Gaussian regression model. In contrast to conventional pretest strategies, that do not dominate the leastsquares (LS) method in terms of meansquared error (MSE), our technique is shown to dominate LS when the e ..."
Abstract
 Add to MetaCart
We develop a pretest type estimator of a deterministic parameter vector β in a linear Gaussian regression model. In contrast to conventional pretest strategies, that do not dominate the leastsquares (LS) method in terms of meansquared error (MSE), our technique is shown to dominate LS when the effective dimension is greater than or equal to 4. Our estimator is based on a simple and intuitive approach in which we first determine the linear minimum MSE (MMSE) estimate that minimizes the MSE. Since the unknown vector β is deterministic, the MSE, and consequently the MMSE solution, will depend in general on β and therefore cannot be implemented. Instead, we propose applying the linear MMSE strategy with the LS substituted for the true value of β to obtain a new estimate. We then use the current estimate in conjunction with the linear MMSE solution to generate another estimate and continue iterating until convergence. As we show, the limit is a pretest type method which is zero when the norm of the data is small, and is otherwise a nonlinear shrinkage of LS.
www.elsevier.com/locate/jspi A pretest like estimator dominating the leastsquares method �
, 2007
"... We develop a pretest type estimator of a deterministic parameter vector β in a linear Gaussian regression model. In contrast to conventional pretest strategies, that do not dominate the leastsquares (LS) method in terms of meansquared error (MSE), our technique is shown to dominate LS when the e ..."
Abstract
 Add to MetaCart
We develop a pretest type estimator of a deterministic parameter vector β in a linear Gaussian regression model. In contrast to conventional pretest strategies, that do not dominate the leastsquares (LS) method in terms of meansquared error (MSE), our technique is shown to dominate LS when the effective dimension is greater than or equal to 4. Our estimator is based on a simple and intuitive approach in which we first determine the linear minimum MSE (MMSE) estimate that minimizes the MSE. Since the unknown vector β is deterministic, the MSE, and consequently the MMSE solution, will depend in general on β and therefore cannot be implemented. Instead, we propose applying the linear MMSE strategy with the LS substituted for the true value of β to obtain a new estimate. We then use the current estimate in conjunction with the linear MMSE solution to generate another estimate and continue iterating until convergence. As we show, the limit is a pretest type method which is zero when the norm of the data is small, and is otherwise a nonlinear shrinkage of LS.