Results 1 
6 of
6
Model Selection by Normalized Maximum Likelihood
, 2005
"... The Minimum Description Length (MDL) principle is an information theoretic approach to inductive inference that originated in algorithmic coding theory. In this approach, data are viewed as codes to be compressed by the model. From this perspective, models are compared on their ability to compress a ..."
Abstract

Cited by 23 (9 self)
 Add to MetaCart
The Minimum Description Length (MDL) principle is an information theoretic approach to inductive inference that originated in algorithmic coding theory. In this approach, data are viewed as codes to be compressed by the model. From this perspective, models are compared on their ability to compress a data set by extracting useful information in the data apart from random noise. The goal of model selection is to identify the model, from a set of candidate models, that permits the shortest description length (code) of the data. Since Rissanen originally formalized the problem using the crude ‘twopart code ’ MDL method in the 1970s, many significant strides have been made, especially in the 1990s, with the culmination of the development of the refined ‘universal code’ MDL method, dubbed Normalized Maximum Likelihood (NML). It represents an elegant solution to the model selection problem. The present paper provides a tutorial review on these latest developments with a special focus on NML. An application example of NML in cognitive modeling is also provided.
Modeling for Optimal Probability Prediction
 In Proceedings of the Nineteenth International Conference on Machine Learning
, 2002
"... We present a general modeling method for optimal probability prediction over future observations, in which model dimensionality is determined as a natural byproduct. This new method yields several estimators, and we establish theoretically that they are optimal (either overall or under stated ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
(Show Context)
We present a general modeling method for optimal probability prediction over future observations, in which model dimensionality is determined as a natural byproduct. This new method yields several estimators, and we establish theoretically that they are optimal (either overall or under stated restrictions) when the number of free parameters is infinite.
A new approach to fitting linear models in high dimensional spaces
, 2000
"... This thesis presents a new approach to fitting linear models, called “pace regression”, which also overcomes the dimensionality determination problem. Its optimality in minimizing the expected prediction loss is theoretically established, when the number of free parameters is infinitely large. In th ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
This thesis presents a new approach to fitting linear models, called “pace regression”, which also overcomes the dimensionality determination problem. Its optimality in minimizing the expected prediction loss is theoretically established, when the number of free parameters is infinitely large. In this sense, pace regression outperforms existing procedures for fitting linear models. Dimensionality determination, a special case of fitting linear models, turns out to be a natural byproduct. A range of simulation studies are conducted; the results support the theoretical analysis. Through the thesis, a deeper understanding is gained of the problem of fitting linear models. Many key issues are discussed. Existing procedures, namely OLS, AIC, BIC, RIC, CIC, CV(d), BS(m), RIDGE, NNGAROTTE and LASSO, are reviewed and compared, both theoretically and empirically, with the new methods. Estimating a mixing distribution is an indispensable part of pace regression. A measurebased minimum distance approach, including probability measures and nonnegative measures, is proposed, and strongly consistent estimators are produced. Of all minimum distance methods for estimating a mixing distribution, only the
Symmetry Properties of BiNormal and BiGamma Receiver Operating Characteristic Curves are Described by KullbackLeibler Divergences
, 2013
"... entropy ..."
(Show Context)
Improving Microarray Spots Segmentation by KMeans driven Adaptive Image Restoration.
"... are used to study human genome. However, microarray images are corrupted by spatially inhomogeneous noise that deteriorates image and consequently gene expression. An adaptive microarray image restoration technique is developed by suitably combining unsupervised clustering with the restoration filte ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
are used to study human genome. However, microarray images are corrupted by spatially inhomogeneous noise that deteriorates image and consequently gene expression. An adaptive microarray image restoration technique is developed by suitably combining unsupervised clustering with the restoration filters for boosting the performance of microarray spots segmentation and for improving the accuracy of subsequent gene expression. Microarray images comprised a publicly available dataset of seven images, obtained from the database of the MicroArray Genome Imaging & Clustering Tool website. Each image contained 6400 spots investigating the diauxic shift of Saccharomyces cerevisiae. The adaptive microarray image restoration technique combined 1/a griding algorithm for locating individual cell images, 2/a clustering algorithm, for assessing local noise from the spot’s background, and 3/a wiener restoration filter, for enhancing individual spots. The effect of the proposed technique quantified using a wellknown boundary detection algorithm (Gradient Vector Flow snake) and the information theoretic metric of Jeffrey’s divergence. The proposed technique increased the Jeffrey’s metric from 0.0194 bits to 0.0314 bits, while boosted the performance of the employed boundary detection algorithm. Application of the proposed technique on cDNA microarray images resulted in noise suppression and facilitated spot edge detection. M I.
Model Selection by Normalized Maximum Likelihood
"... cCorresponding Author The Minimum Description Length (MDL) principle is an information theoretic approach to inductive inference that originated in algorithmic coding theory. In this approach, data are viewed as codes to be compressed by the model. From this perspective, models are compared on thei ..."
Abstract
 Add to MetaCart
(Show Context)
cCorresponding Author The Minimum Description Length (MDL) principle is an information theoretic approach to inductive inference that originated in algorithmic coding theory. In this approach, data are viewed as codes to be compressed by the model. From this perspective, models are compared on their ability to compress a data set by extracting useful information in the data apart from random noise. The goal of model selection is to identify the model, from a set of candidate models, that permits the shortest description length (code) of the data. Since Rissanen originally formalized the problem using the crude ‘twopart code ’ MDL method in the 1970s, many significant strides have been made, especially in the 1990s, with the culmination of the development of the refined ‘universal code ’ MDL method, dubbed Normalized Maximum Likelihood (NML). It represents an elegant solution to the model selection problem. The present paper provides a tutorial review on these latest developments with a special focus on NML. An application example of NML in cognitive modeling is also provided.