Abstract

We exhibit efficient algorithms for agnostic PAC-learning with rectangles, unions of two rectangles, and unions of k intervals as hypotheses. These hypothesis classes are of some interest from the point of view of ap-plied machine learning, because empirical studies show that hypotheses of this simple type (in just one or two of the attributes) provide good prediction rules for various real-world classification problems. In addition, optimal hypotheses of this type may provide valuable heuristic insight into the structure of a real-world classification problem, The algorithms that are introduced in this paper make it feasible to compute optimal hypotheses of this type for a training set of several hundred examples. We also exhibit an approximation algorithm that can compute nearly optimal hypotheses for much larger datasets.

Citations

1515	A Theory of the Learnable - Valiant - 1984
385	Very simple classification rules perform well on most commonly used datasets - Holte - 1993
328	Decision theoretic generalization of the PAC model for neural net and other learning applications - Haussler - 1992
248	Efficient noise-tolerant learning from statistical queries - Kearns - 1998
197	Learning from noisy examples - Angluin, Laird - 1988
182	Efficient distributionfree learning of probabilistic concepts - Kearns, Schapire - 1994
169	Toward efficient agnostic learning - KEARNS, SCHAPIRE, et al. - 1994
158	Learning in the presence of malicious errors - Kearns, Li - 1993
156	An Empirical Comparison of Pruning Methods for Decision–Tree Induction - Mingers - 1989
122	An empirical comparison of pattern recognition, neural nets, and machine learning classi®cation methods - Weiss, Kapouleas - 1989
114	Learning disjunctions of conjunctions - Valiant - 1985
91	A Further Comparison of Splitting Rules for Decision-Tree Induction - Buntine, Niblett - 1992
42	Multi dimensional searching and computational geometry, volume 3 of Data structures and algorithms - Mehlhorn - 1984
37	Statistical queries and faulty PAC oracles - Decatur - 1993
34	Robust trainability of single neurons - Hoeffgen, Simon, et al. - 1993
27	Maximizing the predictive value of production rules - Weiss, Galen, et al. - 1990
4	Computing the rectangle discrepancy - Dobkin, Gunopulos - 1994
3	An empirical comparision of pruning methods for decision tree induction - Mingers - 1989
1	Computing the rectangle discrepancy (video), 3rd Annual Video Review of Computational Geometry - Dobkin, Gunopulos
1	bounds for empirical processes, to appear - Talagrand, Sharper