Results 1 
6 of
6
Shifting: OneInclusion Mistake Bounds and Sample Compression
 EECS DEPARTMENT, UNIVERSITY OF CALIFORNIA, BERKELEY
, 2007
"... ..."
A geometric approach to sample compression
"... The Sample Compression Conjecture of Littlestone & Warmuth has remained unsolved for a quarter century. While maximum classes (concept classes meeting Sauer’s Lemma with equality) can be compressed, the compression of general concept classes reduces to compressing maximal classes (classes that canno ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
The Sample Compression Conjecture of Littlestone & Warmuth has remained unsolved for a quarter century. While maximum classes (concept classes meeting Sauer’s Lemma with equality) can be compressed, the compression of general concept classes reduces to compressing maximal classes (classes that cannot be expanded without increasing VC dimension). Two promising ways forward are: embedding maximal classes into maximum classes with at most a polynomial increase to VC dimension, and compression via operating on geometric representations. This paper presents positive results on the latter approach and a first negative result on the former, through a systematic investigation of finite maximum classes. Simple arrangements of hyperplanes in hyperbolic space are shown to represent maximum classes, generalizing the corresponding Euclidean result. We show that sweeping a generic hyperplane across such arrangements forms an unlabeled compression scheme of size VC dimension and corresponds to a special case of peeling the oneinclusion graph, resolving a recent conjecture of Kuzmin & Warmuth. A bijection between finite maximum classes and certain arrangements of piecewiselinear (PL) hyperplanes in either a ball or Euclidean space is established. Finally we show that dmaximum classes corresponding to PLhyperplane
Recursive Teaching Dimension, Learning Complexity, and Maximum Classes ⋆
"... Abstract. This paper is concerned with the combinatorial structure of concept classes that can be learned from a small number of examples. We show that the recently introduced notion of recursive teaching dimension (RTD, reflecting the complexity of teaching a concept class) is a relevant parameter ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
Abstract. This paper is concerned with the combinatorial structure of concept classes that can be learned from a small number of examples. We show that the recently introduced notion of recursive teaching dimension (RTD, reflecting the complexity of teaching a concept class) is a relevant parameter in this context. Comparing the RTD to selfdirected learning, we establish new lower bounds on the query complexity for a variety of query learning models and thus connect teaching to query learning. For many general cases, the RTD is upperbounded by the VCdimension, e.g., classes of VCdimension 1, (nested differences of) intersectionclosed classes, “standard ” boolean function classes, and finite maximum classes. The RTD thus is the first model to connect teaching to the VCdimension. The combinatorial structure defined by the RTD has a remarkable resemblance to the structure exploited by sample compression schemes and hence connects teaching to sample compression. Sequences of teaching sets defining the RTD coincide with unlabeled compression schemes both (i) resulting from Rubinstein and Rubinstein’s cornerpeeling and (ii) resulting from Kuzmin and Warmuth’s Tail Matching algorithm. 1
Geometric & Topological Representations of Maximum Classes with Applications to Sample Compression
"... We systematically investigate finite maximum classes, which play an important role in machine learning as concept classes meeting Sauer’s Lemma with equality. Simple arrangements of hyperplanes in Hyperbolic space are shown to represent maximum classes, generalizing the corresponding Euclidean resul ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
We systematically investigate finite maximum classes, which play an important role in machine learning as concept classes meeting Sauer’s Lemma with equality. Simple arrangements of hyperplanes in Hyperbolic space are shown to represent maximum classes, generalizing the corresponding Euclidean result. We show that sweeping a generic hyperplane across such arrangements forms an unlabeled compression scheme of size VC dimension and corresponds to a special case of peeling the oneinclusion graph, resolving a conjecture of Kuzmin & Warmuth. A bijection between maximum classes and certain arrangements of PiecewiseLinear (PL) hyperplanes in either a ball or Euclidean space is established. Finally, we show that dmaximum classes corresponding to PL hyperplane arrangements in R d have cubical complexes homeomorphic to a dball, or equivalently complexes that are manifolds with boundary. 1
Shifting, OneInclusion Mistake Bounds . . .
"... Under the prediction model of learning, a prediction strategy is presented with an i.i.d. sample of n − 1 points in X and corresponding labels from a concept f ∈ F, and aims to minimize the worstcase probability of erring on an nth point. By exploiting the structure of F, Haussler et al. achieved a ..."
Abstract
 Add to MetaCart
Under the prediction model of learning, a prediction strategy is presented with an i.i.d. sample of n − 1 points in X and corresponding labels from a concept f ∈ F, and aims to minimize the worstcase probability of erring on an nth point. By exploiting the structure of F, Haussler et al. achieved a VC(F)/n bound for the natural oneinclusion prediction strategy, improving on bounds implied by PACtype results by a O(log n) factor. The key data structure in their result is the natural subgraph of the hypercube—the oneinclusion graph; the key step is a d = VC(F) bound on oneinclusion graph density. The first main result of this paper is a density bound of n � n−1 � n ≤d−1 / ( ≤d) < d, which positively resolves a conjecture of Kuzmin & Warmuth relating to their unlabeled Peeling compression scheme and also leads to an improved mistake bound for the randomized (deterministic) oneinclusion strategy for all d (for d ≈ Θ(n)). The proof uses a new form of VCinvariant shifting and a grouptheoretic symmetrization. Our second main result is a kclass analogue of the d/n mistake bound, replacing the VCdimension by the Pollard pseudodimension and the oneinclusion strategy by its natural hypergraph generalization. This bound on expected risk improves on known PACbased results by a factor of O(log n) and is shown to be optimal up to a O(log k) factor. The combinatorial technique of shifting takes a central role in understanding the oneinclusion (hyper)graph and is a running theme throughout.
Order Compression Schemes ⋆
"... Abstract. Sample compression schemes are schemes for “encoding ” a set of examples in a small subset of examples. The longstanding open sample compression conjecture states that, for any concept class C of VCdimension d, there is a sample compression scheme in which samples for concepts in C are c ..."
Abstract
 Add to MetaCart
Abstract. Sample compression schemes are schemes for “encoding ” a set of examples in a small subset of examples. The longstanding open sample compression conjecture states that, for any concept class C of VCdimension d, there is a sample compression scheme in which samples for concepts in C are compressed to samples of size at most d. We show that every order over C induces a special type of sample compression scheme for C, which we call order compression scheme. It turns out that order compression schemes can compress to samples of size at most d if C is maximum, intersectionclosed, a Dudley class, or of VCdimension 1–and thus in most cases for which the sample compression conjecture is known to be true. Since order compression schemes are much simpler than sample compression schemes in general, their study seems to be a promising step towards resolving the sample compression conjecture. We reveal a number of fundamental properties of order compression schemes, which are helpful in such a study. In particular, order compression schemes exhibit interesting graphtheoretic properties as well as connections to the theory of learning from teachers. 1