Results 1  10
of
17
Shifting: OneInclusion Mistake Bounds and Sample Compression
 EECS DEPARTMENT, UNIVERSITY OF CALIFORNIA, BERKELEY
, 2007
"... ..."
(Show Context)
A geometric approach to sample compression
"... The Sample Compression Conjecture of Littlestone & Warmuth has remained unsolved for a quarter century. While maximum classes (concept classes meeting Sauer’s Lemma with equality) can be compressed, the compression of general concept classes reduces to compressing maximal classes (classes that c ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
The Sample Compression Conjecture of Littlestone & Warmuth has remained unsolved for a quarter century. While maximum classes (concept classes meeting Sauer’s Lemma with equality) can be compressed, the compression of general concept classes reduces to compressing maximal classes (classes that cannot be expanded without increasing VC dimension). Two promising ways forward are: embedding maximal classes into maximum classes with at most a polynomial increase to VC dimension, and compression via operating on geometric representations. This paper presents positive results on the latter approach and a first negative result on the former, through a systematic investigation of finite maximum classes. Simple arrangements of hyperplanes in hyperbolic space are shown to represent maximum classes, generalizing the corresponding Euclidean result. We show that sweeping a generic hyperplane across such arrangements forms an unlabeled compression scheme of size VC dimension and corresponds to a special case of peeling the oneinclusion graph, resolving a recent conjecture of Kuzmin & Warmuth. A bijection between finite maximum classes and certain arrangements of piecewiselinear (PL) hyperplanes in either a ball or Euclidean space is established. Finally we show that dmaximum classes corresponding to PLhyperplane
Recursive Teaching Dimension, Learning Complexity, and Maximum Classes
"... This paper is concerned with the combinatorial structure of concept classes that can be learned from a small number of examples. We show that the recently introduced notion of recursive teaching dimension (RTD, reflecting the complexity of teaching a concept class) is a relevant parameter in this ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
This paper is concerned with the combinatorial structure of concept classes that can be learned from a small number of examples. We show that the recently introduced notion of recursive teaching dimension (RTD, reflecting the complexity of teaching a concept class) is a relevant parameter in this context. Comparing the RTD to selfdirected learning, we establish new lower bounds on the query complexity for a variety of query learning models and thus connect teaching to query learning. For many general cases, the RTD is upperbounded by the VCdimension, e.g., classes of VCdimension 1, (nested differences of) intersectionclosed classes, “standard ” boolean function classes, and finite maximum classes. The RTD thus is the first model to connect teaching to the VCdimension. The combinatorial structure defined by the RTD has a remarkable resemblance to the structure exploited by sample compression schemes and hence connects teaching to sample compression. Sequences of teaching sets defining the RTD coincide with unlabeled compression schemes both (i) resulting from Rubinstein and Rubinstein’s cornerpeeling and (ii) resulting from Kuzmin and Warmuth’s Tail Matching algorithm.
Geometric & Topological Representations of Maximum Classes with Applications to Sample Compression
"... We systematically investigate finite maximum classes, which play an important role in machine learning as concept classes meeting Sauer’s Lemma with equality. Simple arrangements of hyperplanes in Hyperbolic space are shown to represent maximum classes, generalizing the corresponding Euclidean resul ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
(Show Context)
We systematically investigate finite maximum classes, which play an important role in machine learning as concept classes meeting Sauer’s Lemma with equality. Simple arrangements of hyperplanes in Hyperbolic space are shown to represent maximum classes, generalizing the corresponding Euclidean result. We show that sweeping a generic hyperplane across such arrangements forms an unlabeled compression scheme of size VC dimension and corresponds to a special case of peeling the oneinclusion graph, resolving a conjecture of Kuzmin & Warmuth. A bijection between maximum classes and certain arrangements of PiecewiseLinear (PL) hyperplanes in either a ball or Euclidean space is established. Finally, we show that dmaximum classes corresponding to PL hyperplane arrangements in R d have cubical complexes homeomorphic to a dball, or equivalently complexes that are manifolds with boundary. 1
VAPNIKCHERVONENKIS DENSITY ON INDISCERNIBLE SEQUENCES, STABILITY, AND THE MAXIMUM PROPERTY
"... ar ..."
(Show Context)
Feature Selection Techniques and Microarray Data: A Survey
"... Abstract—Feature selection techniques became a lucid want in many bioinformatics applications. Additionally to the massive pool of techniques that have already been developed within the machine learning and data processing fields, specific applications in bioinformatics have crystal rectifier to a w ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract—Feature selection techniques became a lucid want in many bioinformatics applications. Additionally to the massive pool of techniques that have already been developed within the machine learning and data processing fields, specific applications in bioinformatics have crystal rectifier to a wealth of freshly projected techniques. One of the objectives of coming up with feature choice learning algorithms is to get classifiers that rely on a little number of attributes and have verifiable future performance guarantees. There are a unit few, if any, approaches that with success address the two goals at the same time. In this article, we tend to build the interested reader awake to the possibilities of feature selection, providing a basic taxonomy of traditional feature selection techniques, and discussing the premise of conjunction of decision stumps in Occam’s Razor, Sample Compression, and PACBayes learning setting for distinctive a little set of attributes that may be accustomed perform reliable classification tasks. This proposed work presents a brief survey of the feature selection techniques. Keywords—Feature selection, microarray data, decision stump, gene identification. I.
Order Compression Schemes ⋆
"... Abstract. Sample compression schemes are schemes for “encoding ” a set of examples in a small subset of examples. The longstanding open sample compression conjecture states that, for any concept class C of VCdimension d, there is a sample compression scheme in which samples for concepts in C are c ..."
Abstract
 Add to MetaCart
Abstract. Sample compression schemes are schemes for “encoding ” a set of examples in a small subset of examples. The longstanding open sample compression conjecture states that, for any concept class C of VCdimension d, there is a sample compression scheme in which samples for concepts in C are compressed to samples of size at most d. We show that every order over C induces a special type of sample compression scheme for C, which we call order compression scheme. It turns out that order compression schemes can compress to samples of size at most d if C is maximum, intersectionclosed, a Dudley class, or of VCdimension 1–and thus in most cases for which the sample compression conjecture is known to be true. Since order compression schemes are much simpler than sample compression schemes in general, their study seems to be a promising step towards resolving the sample compression conjecture. We reveal a number of fundamental properties of order compression schemes, which are helpful in such a study. In particular, order compression schemes exhibit interesting graphtheoretic properties as well as connections to the theory of learning from teachers. 1
Shifting, OneInclusion Mistake Bounds . . .
"... Under the prediction model of learning, a prediction strategy is presented with an i.i.d. sample of n − 1 points in X and corresponding labels from a concept f ∈ F, and aims to minimize the worstcase probability of erring on an nth point. By exploiting the structure of F, Haussler et al. achieved a ..."
Abstract
 Add to MetaCart
Under the prediction model of learning, a prediction strategy is presented with an i.i.d. sample of n − 1 points in X and corresponding labels from a concept f ∈ F, and aims to minimize the worstcase probability of erring on an nth point. By exploiting the structure of F, Haussler et al. achieved a VC(F)/n bound for the natural oneinclusion prediction strategy, improving on bounds implied by PACtype results by a O(log n) factor. The key data structure in their result is the natural subgraph of the hypercube—the oneinclusion graph; the key step is a d = VC(F) bound on oneinclusion graph density. The first main result of this paper is a density bound of n � n−1 � n ≤d−1 / ( ≤d) < d, which positively resolves a conjecture of Kuzmin & Warmuth relating to their unlabeled Peeling compression scheme and also leads to an improved mistake bound for the randomized (deterministic) oneinclusion strategy for all d (for d ≈ Θ(n)). The proof uses a new form of VCinvariant shifting and a grouptheoretic symmetrization. Our second main result is a kclass analogue of the d/n mistake bound, replacing the VCdimension by the Pollard pseudodimension and the oneinclusion strategy by its natural hypergraph generalization. This bound on expected risk improves on known PACbased results by a factor of O(log n) and is shown to be optimal up to a O(log k) factor. The combinatorial technique of shifting takes a central role in understanding the oneinclusion (hyper)graph and is a running theme throughout.
Study of Feature Selection Techniques using Microarray Data.
"... Abstract—Feature selection techniques became a noticeable need in several bioinformatics applications. In addition to the several techniques that have already been developed among the machine learning and processing fields, specific applications in bioinformatics have more importance to a wealth of ..."
Abstract
 Add to MetaCart
Abstract—Feature selection techniques became a noticeable need in several bioinformatics applications. In addition to the several techniques that have already been developed among the machine learning and processing fields, specific applications in bioinformatics have more importance to a wealth of freshly projected techniques. Using unique feature selection technique for achieving most accurate classifier, for all domains, is next to impossible. Suitable domain specific features can yield better performance for that particular domain. So, we are going to implement Occam’s Razor, Sample Compression, and PACBayes learning algorithms using Decision Stump. We are going to apply these feature selection techniques on microarray data and on the basis of specific efficiency measures we can choose a best suitable technique particularly for microarray data. This paper presents the comparison in between proposed techniques along with traditional feature selection techniques. We apply the projected approaches for gene identification from deoxyribonucleic acid microarray data and compare our results to those of the wellknown triplecrown approaches projected for the task. We tend to show that our algorithmic rule not solely finds hypotheses with a far smaller variety of genes whereas giving competitive classification accuracy, in contrast to different approaches. The projected approaches are general and protrusile in terms of each coming up with novel algorithms and application to different domains. Keywords—Feature selection, microarray data, decision stump, gene identification. I.