Results 1 
5 of
5
The strength of weak learnability
 Machine Learning
, 1990
"... Abstract. This paper addresses the problem of improving the accuracy of an hypothesis output by a learning algorithm in the distributionfree (PAC) learning model. A concept class is learnable (or strongly learnable) if, given access to a Source of examples of the unknown concept, the learner with h ..."
Abstract

Cited by 667 (23 self)
 Add to MetaCart
Abstract. This paper addresses the problem of improving the accuracy of an hypothesis output by a learning algorithm in the distributionfree (PAC) learning model. A concept class is learnable (or strongly learnable) if, given access to a Source of examples of the unknown concept, the learner with high probability is able to output an hypothesis that is correct on all but an arbitrarily small fraction of the instances. The concept class is weakly learnable if the learner can produce an hypothesis that performs only slightly better than random guessing. In this paper, it is shown that these two notions of learnability are equivalent. A method is described for converting a weak learning algorithm into one that achieves arbitrarily high accuracy. This construction may have practical applications as a tool for efficiently converting a mediocre learning algorithm into one that performs extremely well. In addition, the construction has some interesting theoretical consequences, including a set of general upper bounds on the complexity of any strong learning algorithm as a function of the allowed error e.
Learning Simple Concepts Under Simple Distributions
 SIAM JOURNAL OF COMPUTING
, 1991
"... We aim at developing a learning theory where `simple' concepts are easily learnable. In Valiant's learning model, many concepts turn out to be too hard (like NP hard) to learn. Relatively few concept classes were shown to be learnable polynomially. In daily life, it seems that things we care to le ..."
Abstract

Cited by 56 (3 self)
 Add to MetaCart
We aim at developing a learning theory where `simple' concepts are easily learnable. In Valiant's learning model, many concepts turn out to be too hard (like NP hard) to learn. Relatively few concept classes were shown to be learnable polynomially. In daily life, it seems that things we care to learn are usually learnable. To model the intuitive notion of learning more closely, we do not require that the learning algorithm learns (polynomially) under all distributions, but only under all simple distributions. A distribution is simple if it is dominated by an enumerable distrib...
Learning binary relations and total orders
 In Proceedings of the 30th Annual IEEE Symposium on Foundations of Computer Science
, 1989
"... Abstract. We study the problem of designing polynomial prediction algorithms for learning binary relations. We study these problems under an online model in which the instances are drawn by the learner, by a helpful teacher, by an adversary or according to a probability distribution on the instance ..."
Abstract

Cited by 36 (6 self)
 Add to MetaCart
Abstract. We study the problem of designing polynomial prediction algorithms for learning binary relations. We study these problems under an online model in which the instances are drawn by the learner, by a helpful teacher, by an adversary or according to a probability distribution on the instance space. We represent the relation as an n x m binary matrix, and present results for when the matrix is restricted to have at most k distinct row types, and when it is constrained by requiring that the predicate form a total order. 1
Expected Mistake Bound Model for OnLine Reinforcement Learning
 Proceedings of the Fourteenth International Conference on Machine Learning
, 1997
"... We propose a model of efficient online reinforcement learning based on the expected mistake bound framework introduced by Haussler, Littlestone and Warmuth (1987). The measure of performance we use is the expected difference between the total reward received by the learning agent and that received ..."
Abstract

Cited by 12 (1 self)
 Add to MetaCart
We propose a model of efficient online reinforcement learning based on the expected mistake bound framework introduced by Haussler, Littlestone and Warmuth (1987). The measure of performance we use is the expected difference between the total reward received by the learning agent and that received by an agent behaving optimally from the start. We call this expected difference the cumulative mistake of the agent and we require that it "levels off" at a reasonably fast rate as the learning progresses. We show that this model is polynomially equivalent to the PAC model of offline reinforcement learning introduced in (Fiechter, 1994). In particular we show how an offline PAC reinforcement learning algorithm can be transformed into an efficient online algorithm in a simple and practical way. An immediate consequence of this result is that the PAC algorithm for the general finite statespace reinforcement learning problem described in (Fiechter, 1994) can be transformed into a polynomial...
LEARNING BINARY RELATIONS AND TOTAL ORDERS*
"... Abstract. The problem of learning a binary relation between two sets of objects or between a set and itself is studied. This paper represents a binary relation between a set of size n and a set of size rn as an n rn matrix of bits whose (i, j) entry is if and only if the relation holds between the c ..."
Abstract
 Add to MetaCart
Abstract. The problem of learning a binary relation between two sets of objects or between a set and itself is studied. This paper represents a binary relation between a set of size n and a set of size rn as an n rn matrix of bits whose (i, j) entry is if and only if the relation holds between the corresponding elements of the two sets. Polynomial prediction algorithms are presented for learning binary relations in an extended online learning model, where the examples are drawn by the learner, by a helpful teacher, by an adversary, or according to a uniform probability distribution on the instance space. The first part of this paper presents results for the case in which the matrix of the relation has at most k row types. It presents upper and lower bounds on the number of prediction mistakes any prediction algorithm makes when learning such a matrix under the extended online learning model. Furthermore, it describes a technique that simplifies the proof of expected mistake bounds against a randomly chosen query sequence. In the second part of this paper the problem of learning a binary relation that is a total order on a set is considered. A general technique using a fully polynomial randomized approximation scheme (fpras) to implement a randomized version of the halving algorithm is described. This technique is applied to the problem of learning a total order, through the use of an fpras for counting the number of extensions of a partial order, to obtain a polynomial prediction algorithm that with high probability makes at most n lg n + (lg e)lg n mistakes when an adversary selects the query sequence. The case in which a teacher or the learner selects the query sequence is also considered Key words, machine learning, computational learning theory, online learning, mistakebounded learning, binary relations, total orders, fully polynomial randomized approximation schemes AMS subject classifications. 68Q25, 68T05 1. Introduction. In