Getting the most out of ensemble selection

Getting the most out of ensemble selection (2006)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Rich Caruana , Art Munson , Ru Niculescu-mizil

Venue:	In ICDM ’06: Proceedings of the Sixth International Conference on Data Mining
Citations:	10 - 0 self

BibTeX

@INPROCEEDINGS{Caruana06gettingthe,
    author = {Rich Caruana and Art Munson and Ru Niculescu-mizil},
    title = {Getting the most out of ensemble selection},
    booktitle = {In ICDM ’06: Proceedings of the Sixth International Conference on Data Mining},
    year = {2006},
    pages = {828--833},
    publisher = {IEEE Computer Society}
}

Bookmark

OpenURL

Abstract

We investigate four previously unexplored aspects of ensemble selection, a procedure for building ensembles of classifiers. First we test whether adjusting model predictions to put them on a canonical scale makes the ensembles more effective. Second, we explore the performance of ensemble selection when different amounts of data are available for ensemble hillclimbing. Third, we quantify the benefit of ensemble selection’s ability to optimize to arbitrary metrics. Fourth, we study the performance impact of pruning the number of models available for ensemble selection. Based on our results we present improved ensemble selection methods that double the benefit of the original method. 1

Citations

2215	Data Mining: Practical machine learning tools and techniques. 2nd edition - Witten, Frank - 2005
1998	Bagging predictors - Breiman - 1996
785	Random forests - Breiman - 2001
503	Probabilistic outputs for support vector machines and comparison to regularized likelihood methods - Platt - 1999
463	Stacked generalization - Wolpert - 1992
242	The boosting approach to machine learning: an overview - Schapire - 2001
225	T: Analysis and Visualization of Classifier Performance: Comparison Under Imprecise Class and Cost Distributions - Provost, Fawcett
99	T.: Pruning adaptive boosting - Margineantu, Dietterich - 1997
81	Measures of diversity in classifier ensembles and their relationship with ensemble accuracy - Kuncheva, Whitaker - 2003
79	A streaming ensemble algorithm (sea) for large-scale classification - Street, Kim - 2001
57	The Combining Classifier: to Train or Not to Train - DUIN
52	Ensemble selection from libraries of models - Caruana, Niculescu-Mizil, et al. - 2004
50	Tree induction vs. logistic regression: A learning-curve analysis - Perlich, Provost, et al. - 2003
46	Heuristics of instability in model selection - Breiman - 1994
40	Predicting good probabilities with supervised learning - Niculescu-Mizil, Caruana - 2005
39	Combining nearest neighbor classifiers through multiple feature subsets - Bay - 1998
32	Bayesian averaging of classifiers and the overfitting problem - Domingos - 2000
27	Methods for designing multiple classifier systems - Roli, Giacinto, et al. - 2001
18	Ensemble pruning via semi-definite programming - Zhang, Burer, et al.
16	Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets - Bryll, Gutierrez-Osuna, et al. - 2006
15	Applied Data Mining - Giudici - 2003
12	Optimizing to arbitrary NLP metrics using ensemble selection - Munson, Cardie, et al. - 2005
6	Pruning in ordered bagging ensembles - Martínez-Munoz, Suárez - 2006
6	Estimating the potential for combining learning models - Peterson, Martinez
2	Selective fusion of heterogeneous classifiers - Tsoumakas, Angelis, et al.
1	Model compression: Making big, slow models practical - Bucila, Caruana, et al. - 2006