DMCA
Sparse modelling using orthogonal forward regression with press statistic and regularization (2004)
| Venue: | IEEE TRANS. SYSTEMS, MAN AND CYBERNETICS, PART B |
| Citations: | 49 - 23 self |
BibTeX
@ARTICLE{Chen04sparsemodelling,
author = {Sheng Chen and Xia Hong and Chris J. Harris and Paul M. Sharkey},
title = {Sparse modelling using orthogonal forward regression with press statistic and regularization},
journal = {IEEE TRANS. SYSTEMS, MAN AND CYBERNETICS, PART B},
year = {2004},
volume = {34},
pages = {898--911}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
The paper introduces an efficient construction algorithm for obtaining sparse linear-in-the-weights regression models based on an approach of directly optimizing model generalization capability. This is achieved by utilizing the delete-1 cross validation concept and the associated leave-one-out test error also known as the predicted residual sums of squares (PRESS) statistic, without resorting to any other validation data set for model evaluation in the model construction process. Computational efficiency is ensured using an orthogonal forward regression, but the algorithm incrementally minimizes the PRESS statistic instead of the usual sum of the squared training errors. A local regularization method can naturally be incorporated into the model selection procedure to further enforce model sparsity. The proposed algorithm is fully automatic, and the user is not required to specify any criterion to terminate the model construction procedure. Comparisons with some of the existing state-of-art modeling methods are given, and several examples are included to demonstrate the ability of the proposed algorithm to effectively construct sparse models that generalize well.
Keyphrases
orthogonal forward regression press statistic computational efficiency squared training error model generalization capability efficient construction algorithm delete-1 cross validation concept sparse linear-in-the-weights regression model model construction procedure local regularization method validation data set enforce model sparsity predicted residual sum sparse model model construction process model evaluation leave-one-out test error several example state-of-art modeling method usual sum model selection procedure
