Results 1  10
of
16
Transductive Inference for Text Classification using Support Vector Machines
, 1999
"... This paper introduces Transductive Support Vector Machines (TSVMs) for text classification. While regular Support Vector Machines (SVMs) try to induce a general decision function for a learning task, Transductive Support Vector Machines take into account a particular test set and try to minimiz ..."
Abstract

Cited by 825 (4 self)
 Add to MetaCart
This paper introduces Transductive Support Vector Machines (TSVMs) for text classification. While regular Support Vector Machines (SVMs) try to induce a general decision function for a learning task, Transductive Support Vector Machines take into account a particular test set and try to minimize misclassifications of just those particular examples. The paper presents an analysis of why TSVMs are well suited for text classification. These theoretical findings are supported by experiments on three test collections. The experiments show substantial improvements over inductive methods, especially for small training sets, cutting the number of labeled training examples down to a twentieth on some tasks. This work also proposes an algorithm for training TSVMs efficiently, handling 10,000 examples and more.
Optimization Techniques for SemiSupervised Support Vector Machines
"... Due to its wide applicability, the problem of semisupervised classification is attracting increasing attention in machine learning. SemiSupervised Support Vector Machines (S 3 VMs) are based on applying the margin maximization principle to both labeled and unlabeled examples. Unlike SVMs, their fo ..."
Abstract

Cited by 55 (6 self)
 Add to MetaCart
Due to its wide applicability, the problem of semisupervised classification is attracting increasing attention in machine learning. SemiSupervised Support Vector Machines (S 3 VMs) are based on applying the margin maximization principle to both labeled and unlabeled examples. Unlike SVMs, their formulation leads to a nonconvex optimization problem. A suite of algorithms have recently been proposed for solving S 3 VMs. This paper reviews key ideas in this literature. The performance and behavior of various S 3 VM algorithms is studied together, under a common experimental setting.
A continuation method for semisupervised svms
 In International Conference on Machine Learning
, 2006
"... SemiSupervised Support Vector Machines (S3VMs) are an appealing method for using unlabeled data in classification: their objective function favors decision boundaries which do not cut clusters. However their main problem is that the optimization problem is nonconvex and has many local minima, whic ..."
Abstract

Cited by 39 (4 self)
 Add to MetaCart
(Show Context)
SemiSupervised Support Vector Machines (S3VMs) are an appealing method for using unlabeled data in classification: their objective function favors decision boundaries which do not cut clusters. However their main problem is that the optimization problem is nonconvex and has many local minima, which often results in suboptimal performances. In this paper we propose to use a global optimization technique known as continuation to alleviate this problem. Compared to other algorithms minimizing the same objective function, our continuation method often leads to lower test errors. 1.
Branch and Bound for SemiSupervised Support Vector Machines
"... Semisupervised SVMs (S³VM) attempt to learn lowdensity separators by maximizing the margin over labeled and unlabeled examples. The associated optimization problem is nonconvex. To examine the full potential of S3VMs modulo local minima problems in current implementations, we apply branch and bou ..."
Abstract

Cited by 25 (5 self)
 Add to MetaCart
Semisupervised SVMs (S³VM) attempt to learn lowdensity separators by maximizing the margin over labeled and unlabeled examples. The associated optimization problem is nonconvex. To examine the full potential of S3VMs modulo local minima problems in current implementations, we apply branch and bound techniques for obtaining exact, globally optimal solutions. Empirical evidence suggests that the globally optimal solution can return excellent generalization performance in situations where other implementations fail completely. While our current implementation is only applicable to small datasets, we discuss variants that can potentially lead to practically useful algorithms.
Using Labeled and Unlabeled Data to Learn Drifting Concepts
 In Workshop notes of IJCAI01 Workshop on Learning from Temporal and Spatial Data
, 2001
"... For many learning tasks, where data is collected over an extended period of time, one has to cope two problems. The distribution underlying the data is likely to change and only little labeled training data is available at each point in time. A typical example is information filtering, i. e. th ..."
Abstract

Cited by 13 (3 self)
 Add to MetaCart
For many learning tasks, where data is collected over an extended period of time, one has to cope two problems. The distribution underlying the data is likely to change and only little labeled training data is available at each point in time. A typical example is information filtering, i. e. the adaptive classification of documents with respect to a particular user interest. Both the interest of the user and the document content change over time. A filtering system should be able to adapt to such concept changes. Since users often give little feedback, a filtering system should also be able to achieve a good performance, even if only few labeled training examples are provided. This paper proposes a method to recognize and handle concept changes with support vector machines and to use unlabeled data to reduce the need for labeled data. The method maintains windows on the training data, whose size is automatically adjusted so that the estimated generalization error is minimized. The approach is both theoretically wellfounded as well as effective and efficient in practice. Since it does not require complicated parameterization, it is simpler to use and more robust than comparable heuristics. Experiments with simulated concept drift scenarios based on realworld text data compare the new method with other window management approaches and show that it can effectively select an appropriate window size in a robust way. In order to achieve an acceptable performance with fewer labeled training examples, the proposed method exploits unlabeled examples in a transductive way. 1
ON SEMISUPERVISED KERNEL METHODS
"... Semisupervised learning is an emerging computational paradigm for learning from limited supervision by utilizing large amounts of inexpensive, unsupervised observations. Not only does this paradigm carry appeal as a model for natural learning, but it also has an increasing practical need in most if ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
Semisupervised learning is an emerging computational paradigm for learning from limited supervision by utilizing large amounts of inexpensive, unsupervised observations. Not only does this paradigm carry appeal as a model for natural learning, but it also has an increasing practical need in most if not all applications of machine learning – those where abundant amounts of data can be cheaply and automatically collected but manual labeling for the purposes of training learning algorithms is often slow, expensive, and errorprone. In this thesis, we develop families of algorithms for semisupervised inference. These algorithms are based on intuitions about the natural structure and geometry of probability distributions that underlie typical datasets for learning. The classical framework of Regularization in Reproducing Kernel Hilbert Spaces (which is the basis of stateoftheart supervised algorithms such as SVMs) is extended in several ways to utilize unlabeled data. These extensions are embodied in the following contributions: (1) Manifold Regularization is based on the assumption that highdimensional
SemiSupervised Learning in Initially Labeled NonStationary Environments with Gradual Drift
 International Joint Conference on Neural Networks (IJCNN 2012
, 2012
"... Abstract—Semisupervised learning (SSL) in nonstationary environments has received relatively little attention in machine learning, despite a growing number of applications that can benefit from a properly configured SSL algorithm. Previous works in learning nonstationary data have analyzed such c ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
Abstract—Semisupervised learning (SSL) in nonstationary environments has received relatively little attention in machine learning, despite a growing number of applications that can benefit from a properly configured SSL algorithm. Previous works in learning nonstationary data have analyzed such cases where both labeled and unlabeled instances are received at every time step and/or in regular intervals; however, to the best of our knowledge, no work has investigated the case where labeled instances are received only at the initial time step, followed by unlabeled instances provided in subsequent time steps. In this proofofconcept work, we propose a new framework for learning in a nonstationary environment that provides only unlabeled data after the initial time step, to which we refer to as initially labeled environment. The proposed framework generates labels for previously unlabeled data at each time step to be combined with incoming unlabeled data – possibly from a drifting distribution using a compacted polytope sample extraction algorithm. We have conducted two experiments to demonstrate the feasibility and reliability of the approach. This proofofconcept is presented in two dimensions; however, the algorithm can be extended to higher dimensions with appropriate modifications. Keywordsalpha shape; concept drift; nonstationary environment; shape offsets; semisupervised learning I.
Infinitesimal Annealing for Training SemiSupervised Support Vector Machines
"... The semisupervised support vector machine (S3VM) is a maximummargin classification algorithm based on both labeled and unlabeled data. Training S3VM involves either a combinatorial or nonconvex optimization problem and thus finding the global optimal solution is intractable in practice. It has be ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
The semisupervised support vector machine (S3VM) is a maximummargin classification algorithm based on both labeled and unlabeled data. Training S3VM involves either a combinatorial or nonconvex optimization problem and thus finding the global optimal solution is intractable in practice. It has been demonstrated that a key to successfully find a good (local) solution of S3VM is to gradually increase the effect of unlabeled data, à la annealing. However, existing algorithms suffer from the tradeoff between the resolution of annealing steps and the computation cost. In this paper, we go beyond this tradeoff by proposing a novel training algorithm that efficiently performs annealing with an infinitesimal resolution. Through experiments, we demonstrate that the proposed infinitesimal annealing algorithm tends to produce better solutions with less computation time than existing approaches. 1.
COMPOSE: A SemiSupervised Learning Framework for Initially Labeled Nonstationary Streaming Data
"... Abstract – An increasing number of realworld applications are associated with streaming data drawn from drifting and nonstationary distributions that change over time. These applications demand new algorithms that can learn and adapt to such changes, also known as concept drift. Proper characteriza ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
Abstract – An increasing number of realworld applications are associated with streaming data drawn from drifting and nonstationary distributions that change over time. These applications demand new algorithms that can learn and adapt to such changes, also known as concept drift. Proper characterization of such data with existing approaches typically requires substantial amount of labeled instances, which may be difficult, expensive or even impractical to obtain. In this contribution, we introduce COMPOSE, a computational geometry based framework to learn from nonstationary streaming data, where labels are unavailable (or presented very sporadically) after initialization. We introduce the algorithm in detail, and discuss its results and performances on several synthetic and realworld datasets, which demonstrate the ability of the algorithm to learn under several different scenarios of initially labeled streaming environments (ILSE). On carefully designed synthetic datasets, we compare the performance of COMPOSE against the optimal Bayes classifier, as well as the APT algorithm, which addresses a similar environment referred to as extreme verification latency. Furthermore, using the realworld NOAA Weather Dataset, we demonstrate that COMPOSE is competitive even with a wellestablished, fully supervised, nonstationary learning algorithm that receives labeled data in every batch.
Some Contributions to SemiSupervised Learning
, 2010
"... Semisupervised learning methods attempt to improve the performance of a supervised or an unsupervised learner in the presence of “side information”. This side information can be in the form of unlabeled samples in the supervised case or pairwise constraints in the unsupervised case. Most existing ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Semisupervised learning methods attempt to improve the performance of a supervised or an unsupervised learner in the presence of “side information”. This side information can be in the form of unlabeled samples in the supervised case or pairwise constraints in the unsupervised case. Most existing semisupervised learning approaches design a new objective function, which in turn leads to a new algorithm rather than improving the performance of an already available learner. In this thesis, the three classical problems in pattern recognition and machine learning, namely, classification, clustering, and unsupervised feature selection, are extended to their semisupervised counterparts. Our first contribution is an algorithm that utilizes unlabeled data along with the labeled data while training classifiers. Unlike previous approaches that design specialized algorithms to effectively exploit the labeled and unlabeled data, we design a metasemisupervised learning algorithm called SemiBoost, which wraps around the underlying supervised algorithm and improve its performance using the unlabeled data and a similarity function. Empirical evaluation on several standard datasets shows a significant improvement in the performance of wellknown classifiers (decision stump, decision tree, and SVM).