Results 1  10
of
13
Bayesian Ying Yang system, best harmony learning, and Gaussian manifold based family
 Computational Intelligence: Research Frontiers, WCCI2008 Plenary/Invited Lectures. Lecture Notes in Computer Science
"... five action circling ..."
A LargeSample Model Selection Criterion Based on Kullback's Symmetric Divergence
 Statistical and Probability Letters
, 1999
"... The Akaike information criterion, AIC, is a widely known and extensively used tool for statistical model selection. AIC serves as an asymptotically unbiased estimator of a variant of Kullback's directed divergence between the true model and a fitted approximating model. The directed divergence is an ..."
Abstract

Cited by 11 (1 self)
 Add to MetaCart
The Akaike information criterion, AIC, is a widely known and extensively used tool for statistical model selection. AIC serves as an asymptotically unbiased estimator of a variant of Kullback's directed divergence between the true model and a fitted approximating model. The directed divergence is an asymmetric measure of separation between two statistical models, meaning that an alternate directed divergence may be obtained by reversing the roles of the two models in the definition of the measure. The sum of the two directed divergences is Kullback's symmetric divergence. Since the symmetric divergence combines the information in two related though distinct measures, it functions as a gauge of model disparity which is arguably more sensitive than either of its individual components. With this motivation, we propose a model selection criterion which serves as an asymptotically unbiased estimator of a variant of the symmetric divergence between the true model and a fitted approximating model. We examine the performance of the criterion relative to other wellknown criteria in a simulation study. Keywords: AIC, Akaike information criterion, Idivergence, Jdivergence, KullbackLeibler information, relative entropy. Correspondence: Joseph E. Cavanaugh, Department of Statistics, 222 Math Sciences Bldg., University of Missouri, Columbia, MO 65211. y This research was supported by NSF grant DMS9704436. 1.
An improved Akaike Information Criterion for Statespace Model Selection
 Comput. Stat. Data An
, 2006
"... Following the work of Hurvich, Shumway, and Tsai (1990), we propose an “improved ” variant of the Akaike information criterion, AICi, for statespace model selection. The variant is based on Akaike’s (1973) objective of estimating the KullbackLeibler information (Kullback 1968) between the densitie ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
Following the work of Hurvich, Shumway, and Tsai (1990), we propose an “improved ” variant of the Akaike information criterion, AICi, for statespace model selection. The variant is based on Akaike’s (1973) objective of estimating the KullbackLeibler information (Kullback 1968) between the densities corresponding to the fitted model and the generating or true model. The development of AICi proceeds by decomposing the expected information into two terms. The first term suggests that the empirical log likelihood can be used to form a biased estimator of the information; the second term provides the bias adjustment. Exact computation of the bias adjustment requires the values of the true model parameters, which are inaccessible in practical applications. Yet for fitted models in the candidate class that are correctly specified or overfit, the adjustment is asymptotically independent of the true parameters. Thus, in certain settings, the adjustment may be estimated via Monte Carlo simulations by using conveniently chosen simulation parameters as proxies for the true parameters. We present simulation results to evaluate the performance of AICi both as an estimator of the KullbackLeibler information and as a model selection criterion. Our results indicate that AICi estimates the information with less bias than traditional AIC. Furthermore, AICi serves as an effective tool for selecting a model of appropriate dimension. Keywords: AIC, KullbackLeibler information, Kullback’s directed divergence, statespace model, time series analysis.
Parsimonious segmentation of time series’ by Potts models
 Innovations in Classification, Data Science, and Information Systems. Proc. 27th Annual GfKl Conference, University of Cottbus, March 12  14, 2003., Studies in Classification, Data Analysis, and Knowledge Organization
, 2004
"... Abstract. Typical problems in the analysis of data sets like timeseries or images crucially rely on the extraction of primitive features based on segmentation. Variational approaches are a popular and convenient framework in which such problems can be studied. We focus on Potts models as simple non ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
Abstract. Typical problems in the analysis of data sets like timeseries or images crucially rely on the extraction of primitive features based on segmentation. Variational approaches are a popular and convenient framework in which such problems can be studied. We focus on Potts models as simple nontrivial instances. The discussion proceeds along two data sets from brain mapping and functional genomics. 1
Don’t shed tears over breaks
 DMV Nachrichten
, 2005
"... imaging Mathematical Subject Classification: 93E14, 62G08, 68T45, 49M20, 90C31 This essay deals with ‘discontinuous phenomena ’ in timeseries. It is an introduction to, and a brief survey of aspects concerning the concepts of segmentation into ‘smooth ’ pieces on the one hand, and the complementary ..."
Abstract

Cited by 4 (4 self)
 Add to MetaCart
imaging Mathematical Subject Classification: 93E14, 62G08, 68T45, 49M20, 90C31 This essay deals with ‘discontinuous phenomena ’ in timeseries. It is an introduction to, and a brief survey of aspects concerning the concepts of segmentation into ‘smooth ’ pieces on the one hand, and the complementary notion of the identification of jumps, on the other hand. We restrict ourselves to variational approaches, both in discrete, and in continuous time. They will define ‘filters’, with data as ‘inputs ’ and minimizers of functionals as ‘outputs’. The main example is a particularly simple model, which, for historical reasons, we decided to call the Potts functional. We will argue that it is an appropriate tool for the extraction of the simplest and most basic morphological features from data. This is an attempt to interpret data from a welldefined point of view. It is in contrast to restoration of a true signal perhaps distorted and degraded by noise which is not in the main focus of this paper.
A Modified Information Criterion for Cointegration Tests based on a VAR Approximation
, 2006
"... We consider Johansen’s (1988, 1991) cointegration tests when a Vector AutoRegressive (VAR) process of order k is used to approximate a more general linear process with a possibly infinite VAR representation. Traditional methods to select the lag order, such as Akaike’s (AIC) or the Bayesian informat ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
We consider Johansen’s (1988, 1991) cointegration tests when a Vector AutoRegressive (VAR) process of order k is used to approximate a more general linear process with a possibly infinite VAR representation. Traditional methods to select the lag order, such as Akaike’s (AIC) or the Bayesian information criteria, often lead to too parsimonious a model with the implication that the cointegration tests suffer from substantial size distortions in finite samples. We extend the analysis of Ng and Perron (2001) to derive a Modified Akaike’s Information Criterion (MAIC) in this multivariate setting. The idea is to use the information specified by the null hypothesis as it relates to restrictions on the parameters of the model to keep an extra term in the penalty function of the AIC. This MAIC takes a very simple form for which this extra term is simply the likelihood ratio test for testing the null hypothesis of r against more than r cointegrating vectors. We provide theoretical analyses of its validity and of the fact that cointegration tests constructed from a VAR whose lag order is selected using the MAIC have the same limit distribution as when the order is finite and known. We also provide theoretical and simulation analyses to show how the MAIC leads to VAR approximations that yield tests with drastically improved size properties with little loss of power.
Criteria for Linear Model Selection Based on Kullback's Symmetric Divergence
"... imulation study. Our results indicate that the new criteria perform favorably against their AIC analogues. Key words: AIC, Akaike information criterion, Idivergence, Jdivergence, KullbackLeibler information, regression, relative entropy. Acknowledgments. The author is indebted to the editor, ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
imulation study. Our results indicate that the new criteria perform favorably against their AIC analogues. Key words: AIC, Akaike information criterion, Idivergence, Jdivergence, KullbackLeibler information, regression, relative entropy. Acknowledgments. The author is indebted to the editor, the associate editor, and two referees whose thoughtful suggestions helped to improve the original version of this manuscript. This research was supported by the National Science Foundation, grant DMS9704436. 1. Introduction An important component of any linear modeling problem consists of determining an appropriate size and form for the design matrix. Improper specification may substantially impact both estimators of the model parameters and predictors of the response variable: underspecification may lead to results which are severely biased, whereas overspecification may lead to results with unnecessarily high variability. Model selection criteria, such as the Akaike (1973) informatio
Models with Delays for Cell Population Dynamics: Identification, Selection and Analysis. Part I: Computational Modelling with Functional Differential Equations: Identification, Selection, and Sensitivity
, 2003
"... Parameter identifiability is concerned with the question whether the parameters of a specific model can be identified from knowledge about certain solutions of the model, assuming perfect data." (That the data is discretely defined is subsequently apparent.) From the perspective adopted in our paper ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Parameter identifiability is concerned with the question whether the parameters of a specific model can be identified from knowledge about certain solutions of the model, assuming perfect data." (That the data is discretely defined is subsequently apparent.) From the perspective adopted in our paper, identifiability is more concerned with whether there is an identifiable model from amongst a hierarchy of models that optimises some measurement of best fit to noisy data (a measurement that may include an index reflecting complexity or parsimony). Whilst the analysis of identifiability within one scenario may provide insight in the context of a different scenario, one should be clear about which problem is under consideration at any given time.
BBY Harmony Learning, Structural RPCL, . . .
, 2002
"... The Bayesian YingYang (BYY) harmony learning acts as a general statistical learning framework, featured by not only new regularization techniques for parameter learning but also a new mechanism that implements model selection either automatically during parameter learning or via a new class of mode ..."
Abstract
 Add to MetaCart
The Bayesian YingYang (BYY) harmony learning acts as a general statistical learning framework, featured by not only new regularization techniques for parameter learning but also a new mechanism that implements model selection either automatically during parameter learning or via a new class of model selection criteria used after parameter learning. In this paper, further advances on BYY harmony learning by considering modular inner representations are presented in three parts. One consists of results on unsupervised mixture models, ranging from Gaussian mixture based Mean Square Error (MSE) clustering, elliptic clustering, subspace clustering to NonGaussian mixture based clustering not only with each cluster represented via either Bernoulli  Gaussian mixtures or independent real factor models, but also with independent component analysis implicitly made on each cluster. The second consists of results on supervised mixtureofexperts (ME) models, including Gaussian ME, Radial Basis Function nets, and Kernel regressions. The third consists of two strategies for extending the above structural mixtures into selforganized topological maps. All these advances are introduced with details on three issues, namely, (a) adaptive learning algorithms, especially elliptic, subspace, and structural rival penalized competitive learning algorithms, with model selection made automatically during learning; (b) model selection criteria for being used after parameter learning, and (c) how these learning algorithms and criteria are obtained from typical special cases of BYY harmony learning.