On the convergence of leveraging

On the convergence of leveraging (2002)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Gunnar Rätsch , Sebastian Mika , Manfred K. Warmuth

Venue:	In Advances in Neural Information Processing Systems (NIPS
Citations:	7 - 2 self

BibTeX

@INPROCEEDINGS{Rätsch02onthe,
    author = {Gunnar Rätsch and Sebastian Mika and Manfred K. Warmuth},
    title = {On the convergence of leveraging},
    booktitle = {In Advances in Neural Information Processing Systems (NIPS},
    year = {2002},
    publisher = {MIT Press}
}

Years of Citing Articles

Bookmark

OpenURL

Abstract

We give an unified convergence analysis of ensemble learning methods including e.g. AdaBoost, Logistic Regression and the Least-Square-Boost algorithm for regression. These methods have in common that they iteratively call a base learning algorithm which returns hypotheses that are then linearly combined. We show that these methods are related to the Gauss-Southwell method known from numerical optimization and state non-asymptotical convergence results for all these methods. Our analysis includes ℓ1-norm regularized cost functions leading to a clean and general way to regularize ensemble learning. 1

Citations

6696	The Nature of Statistical Learning Theory - Vapnik - 1995
2333	Convex Analysis - Rockafellar - 1970
1714	Schapire R: A decision-theoretic generalization of online learning and an application to boosting - Freund - 1997
1512	Smola A: Learning with Kernels - Schölkopf - 2002
1089	Atomic decomposition by basis pursuit - Chen, Donoho, et al. - 1998
896	Additive logistic regression: a statistical view of boosting - Friedman, Hastie, et al. - 2000
561	Y: Improved boosting algorithms using confidence-rated predictions - RE, Singer - 1999
554	Nonlinear Programming. Athena Scientific - Bertsekas - 1995
464	Inducing Features of Random Fields - Pietra, Pietra, et al. - 1997
199	Soft margins for AdaBoost - Ratsch, Onoda, et al. - 1998
171	Y.: Logistic regression, AdaBoost and Bregman distances - Collins, Schapire, et al. - 2002
122	Functional gradient techniques for combining hypotheses - Mason, Baxter, et al. - 2000
113	Prediction games and arcing algorithms - Breiman - 1997
51	Boosting as entropy projection - Kivinen, Warmuth - 1999
49	A mathematical programming approach to the kernel fisher algorithm - Mika, Rätsch, et al. - 2001
40	On the convergence of coordinate descent method for convex differentiable minimization - Luo, Tseng - 1992
39	Convex analysis. Princeton Landmarks in Mathematics - Rockafellar - 1997
36	Legendre functions and the method of random Bregman projections - Bauschke, Borwein - 1997
34	Nonlinear Programming, Athena Scienti c - Bertsekas - 1995
32	A column generation algorithm for boosting - Bennett, Demiriz, et al. - 2000
32	Improved boosting algorithms using con predictions - Schapire, Singer - 1998
24	Robust ensemble learning - Ratsch, Schokopf, et al. - 1999
23	Lipschitz continuity of solutions of linear inequalities, programs and complementarity problems - Mangasarian - 1987
23	Duality and auxiliary functions for Brègman distances - Pietra, Pietra, et al. - 2002
22	Inducing features of random - Pietra, S, et al. - 1997
20	A geometric approach to leveraging weak learners - Duffy, Helmbold - 1999
19	Regression Selection and Shrinkage Via the Lasso - Tibshirani - 1995
18	Parsimonious Least Norm Approximation - Bradley, Mangasarian - 1998
18	Regression, prediction, shrinkage - Copas - 1983
18	Leveraging for regression - Duffy, Helmbold - 2000
17	Potential boosters - Duffy, Helmbold - 1999
17	Robust boosting via convex optimization - Rätsch - 2001
14	Bounds for error in the solution set of a perturbed linear program - ROBINSON - 1973
11	An asymptotic analysis of AdaBoost in the binary classification case - Onoda, Rätsch, et al. - 1998
11	Sparse regression ensembles in infinite and finite hypothesis spaces - Rätsch, Demiriz, et al. - 2002
10	Bounds on approximate steepest descent for likelihood maximization in exponential families - Cesa-Bianchi, Krogh, et al. - 1994
9	Leveraged vector machines - Singer - 2000
8	Greedy function approximation - Friedman - 1999
7	and Nonlinear Programming. Addison-Wesley Publishing Co - Linear - 1984
6	A general greedy approximation algorithm with applications, in - Zhang - 2002
6	On the convergence of leveraging. NeuroCOLT2 - Rätsch, Mika, et al. - 2001
2	Sparse regression ensembles in in and hypothesis spaces - Ratsch, Demiriz, et al. - 2000
1	On approximate solutions of systems of linear inequalities - Homann - 1952
1	On approximate solutions of systems of linear inequalities - Hoffmann - 1952