Query by Committee (1992) [169 citations — 2 self]
Popular Tags
No tags have been applied to this document.
BibTeX | Add To MetaCart
author = {H. S. Seung and M. Opper and H. Sompolinsky},
title = {Query by Committee},
year = {1992}
}
Bookmarks
OpenURL
Abstract:
We propose an algorithm called query by committee, in which a committee of students is trained on the same data set. The next query is chosen according to the principle of maximal disagreement. The algorithm is studied for two toy models: the high-low game and perceptron learning of another perceptron. As the number of queries goes to infinity, the committee algorithm yields asymptotically finite information gain. This leads to generalization error that decreases exponentially with the number of examples. This in marked contrast to learning from randomly chosen inputs, for which the information gain approaches zero and the generalization error decreases with a relatively slow inverse power law. We suggest that asymptotically finite information gain may be an important characteristic of good query algorithms.
Citations
| 187 | Theory of Optimal Experiments – Fedorov - 1972 |
| 88 | Bounds on the sample complexity of Bayesian learning using information theory and the VC dimension – Haussler, Kearns, et al. - 1994 |
| 54 | Neural net algorithms that learn in polynomial time from examples and queries – Baum - 1991 |
| 41 | Statistical mechanics of learning from examples – Seung, Sompolinsky, et al. - 1992 |
| 38 | Consistent inference on probabilities in layered networks, predictions and generalization – Tishby, Levin - 1989 |
| 24 | Statistical theory of learning a rule – Gyorgyi, Tishby - 1990 |
| 24 | Generalization performance of bayes optimal classification algorithm for learning a perceptron – Opper, Haussler - 1991 |
| 21 | Improving a network generalization ability by selecting examples – Kinzel - 1990 |
| 13 | Selecting examples for perceptrons – Watkin, Rau - 1992 |
| 12 | Three unfinished works on the optimal storage capacity of networks – Gardner, Derrida - 1989 |
