CiteSeerX — A Decision-Theoretic Generalization of on-Line Learning and an Application to Boosting

A Decision-Theoretic Generalization of on-Line Learning and an Application to Boosting (1995) [1205 citations — 40 self]

Popular Tags

No tags have been applied to this document.

BibTeX | Add To MetaCart

@MISC{Freund95adecision-theoretic,
    author = {Yoav Freund and Robert E. Schapire},
    title = {A Decision-Theoretic Generalization of on-Line Learning and an Application to Boosting},
    year = {1995}
}

Bookmarks

OpenURL

Abstract:

In the first part of the paper we consider the problem of dynamically apportioning resources among a set of options in a worst-case on-line framework. The model we study can be interpreted as a broad, abstract extension of the well-studied on-line prediction model to a general decision-theoretic setting. We show that the multiplicative weightupdate rule of Littlestone and Warmuth [15] can be adapted to this model yielding bounds that are slightly weaker in some cases, but applicable to a considerably more general class of learning problems. We show how the resulting learning algorithm can be applied to a variety of problems, including gambling, multiple-outcome prediction, repeated games and prediction of points in R n . In the second part of the paper we apply the multiplicative weight-update technique to derive a new boosting algorithm. This boosting algorithm does not require any prior knowledge about the performance of the weak learning algorithm. We also study generalizations of...

Citations

1045	Experiments with a new boosting algorithm – Freund, Schapire - 1996
624	Estimation of Dependences Based on Empirical Data – Vapnik - 1982
457	The strength of weak learnability – Schapire - 1990
438	The weighted majority algorithm – Littlestone, Warmuth - 1994
422	An Introduction to Computational Learning Theory – Kearns, Vazirani - 1994
294	Boosting a Weak Learning Algorithm by Majority – Freund - 1995
281	What size net gives valid generalization – Baum, Haussler - 1989
228	How to use expert advice – Cesa-Bianchi, Freund, et al. - 1997
222	Bagging, boosting, and C4.5 – Quinlan - 1996
180	Aggregating strategies – Vovk - 1990
98	Approximation to Bayes risk in repeated play – Hannan - 1957
94	Universal portfolios – Cover - 1991
94	Game theory, on-line prediction, and boosting – Freund, Schapire - 1996
91	An analog of the minimax theorem for vector payoffs – Blackwell - 1956
86	An experimental and theoretical comparison of model selection methods – Kearns, Mansour, et al. - 1997
78	Boosting decision trees – Drucker, Cortes - 1996
71	A game of prediction with expert advice – Vovk - 1998
68	Boosting performance in neural networks – Drucker, Schapire, et al. - 1993
47	Tight worst-case loss bounds for predicting with expert advice – Haussler, Kivinen, et al. - 1995
38	Dietterich and Ghulum Bakiri. Solving multiclass learning problems via errorcorrecting output codes – Thomas - 1995
19	Learning sparse perceptrons – Jackson, Craven - 1996
16	Data Filtering and Distribution Modeling Algorithms for Machine Learning – Freund - 1993
13	Using experts for predicting continuous outcomes – Kivinen, Warmuth - 1994
12	Approximate methods for sequential decision making using expert advice – Chung - 1994
8	How to use expert advice – Schapire, Warmuth - 1993
8	Boosting a weak learning algorithm by majority. Information and Computation, To appear. An extended abstract appeared – Freund - 1990
7	editors, Contributions to the Theory – Tucker, Wolfe - 1957
5	arcing classifiers. Unpublished manuscript – Bias - 1996
1	Boosting decision trees. Unpublished manuscript – Drucker, Cortes - 1995