Results 1  10
of
93
SemiSupervised Learning Literature Survey
, 2006
"... We review the literature on semisupervised learning, which is an area in machine learning and more generally, artificial intelligence. There has been a whole
spectrum of interesting ideas on how to learn from both labeled and unlabeled data, i.e. semisupervised learning. This document is a chapter ..."
Abstract

Cited by 757 (8 self)
 Add to MetaCart
(Show Context)
We review the literature on semisupervised learning, which is an area in machine learning and more generally, artificial intelligence. There has been a whole
spectrum of interesting ideas on how to learn from both labeled and unlabeled data, i.e. semisupervised learning. This document is a chapter excerpt from the author’s
doctoral thesis (Zhu, 2005). However the author plans to update the online version frequently to incorporate the latest development in the field. Please obtain the latest
version at http://www.cs.wisc.edu/~jerryzhu/pub/ssl_survey.pdf
Label propagation through linear neighborhoods
 ICML06, 23rd International Conference on Machine Learning
, 2006
"... A novel semisupervised learning approach is proposed based on a linear neighborhood model, which assumes that each data point can be linearly reconstructed from its neighborhood. Our algorithm, named Linear Neighborhood Propagation (LNP), can propagate the labels from the labeled points to the whol ..."
Abstract

Cited by 107 (13 self)
 Add to MetaCart
(Show Context)
A novel semisupervised learning approach is proposed based on a linear neighborhood model, which assumes that each data point can be linearly reconstructed from its neighborhood. Our algorithm, named Linear Neighborhood Propagation (LNP), can propagate the labels from the labeled points to the whole dataset using these linear neighborhoods with sufficient smoothness. We also derive an easy way to extend LNP to outofsample data. Promising experimental results are presented for synthetic data, digit and text classification tasks. 1.
Graph construction and bmatching for semisupervised learning
 In International Conference on Machine Learning
"... Graph based semisupervised learning (SSL) methods play an increasingly important role in practical machine learning systems. A crucial step in graph based SSL methods is the conversion of data into a weighted graph. However, most of the SSL literature focuses on developing label inference algorithm ..."
Abstract

Cited by 62 (12 self)
 Add to MetaCart
(Show Context)
Graph based semisupervised learning (SSL) methods play an increasingly important role in practical machine learning systems. A crucial step in graph based SSL methods is the conversion of data into a weighted graph. However, most of the SSL literature focuses on developing label inference algorithms without extensively studying the graph building method and its effect on performance. This article provides an empirical study of leading semisupervised methods under a wide range of graph construction algorithms. These SSL inference algorithms include the Local and Global Consistency (LGC) method, the Gaussian Random Field (GRF) method, the Graph Transduction via Alternating Minimization (GTAM) method as well as other techniques. Several approaches for graph construction, sparsification and weighting are explored including the popular knearest neighbors method (kNN) and the bmatching method. As opposed to the greedily constructed kNN graph, the bmatched graph ensures each node in the graph has the same number of edges and produces a balanced or regular graph. Experimental results on both artificial data and real benchmark datasets indicate that bmatching produces more robust graphs and therefore provides significantly better prediction accuracy without any significant change in computation time.
Deep Learning from Temporal Coherence in Video
"... This work proposes a learning method for deep architectures that takes advantage of sequential data, in particular from the temporal coherence that naturally exists in unlabeled video recordings. That is, two successive frames are likely to contain the same object or objects. This coherence is used ..."
Abstract

Cited by 46 (2 self)
 Add to MetaCart
(Show Context)
This work proposes a learning method for deep architectures that takes advantage of sequential data, in particular from the temporal coherence that naturally exists in unlabeled video recordings. That is, two successive frames are likely to contain the same object or objects. This coherence is used as a supervisory signal over the unlabeled data, and is used to improve the performance on a supervised task of interest. We demonstrate the effectiveness of this method on some pose invariant object and face recognition tasks. 1.
Semisupervised Multilabel Learning by Solving a Sylvester Equation
"... Multilabel learning refers to the problems where an instance can be assigned to more than one category. In this paper, we present a novel Semisupervised algorithm for Multilabel learning by solving a Sylvester Equation (SMSE). Two graphs are first constructed on instance level and category level ..."
Abstract

Cited by 45 (0 self)
 Add to MetaCart
(Show Context)
Multilabel learning refers to the problems where an instance can be assigned to more than one category. In this paper, we present a novel Semisupervised algorithm for Multilabel learning by solving a Sylvester Equation (SMSE). Two graphs are first constructed on instance level and category level respectively. For instance level, a graph is defined based on both labeled and unlabeled instances, where each node represents one instance and each edge weight reflects the similarity between corresponding pairwise instances. Similarly, for category level, a graph is also built based on
Semisupervised regression with cotraining style algorithms
, 2007
"... The traditional setting of supervised learning requires a large amount of labeled training examples in order to achieve good generalization. However, in many practical applications, unlabeled training examples are readily available but labeled ones are fairly expensive to obtain. Therefore, semisup ..."
Abstract

Cited by 43 (8 self)
 Add to MetaCart
(Show Context)
The traditional setting of supervised learning requires a large amount of labeled training examples in order to achieve good generalization. However, in many practical applications, unlabeled training examples are readily available but labeled ones are fairly expensive to obtain. Therefore, semisupervised learning has attracted much attention. Previous research on semisupervised learning mainly focuses on semisupervised classification. Although regression is almost as important as classification, semisupervised regression is largely understudied. In particular, although cotraining is a main paradigm in semisupervised learning, few works has been devoted to cotraining style semisupervised regression algorithms. In this paper, a cotraining style semisupervised regression algorithm, i.e. COREG, is proposed. This algorithm uses two regressors each labels the unlabeled data for the other regressor, where the confidence in labeling an unlabeled example is estimated through the amount of reduction in mean square error over the labeled neighborhood of that example. Analysis and experiments show that COREG can effectively exploit unlabeled data to improve regression estimates.
Statistical Analysis of SemiSupervised Regression
"... Semisupervised methods use unlabeled data in addition to labeled data to construct predictors. While existing semisupervised methods have shown some promising empirical performance, their development has been based largely based on heuristics. In this paper we study semisupervised learning from t ..."
Abstract

Cited by 41 (1 self)
 Add to MetaCart
Semisupervised methods use unlabeled data in addition to labeled data to construct predictors. While existing semisupervised methods have shown some promising empirical performance, their development has been based largely based on heuristics. In this paper we study semisupervised learning from the viewpoint of minimax theory. Our first result shows that some common methods based on regularization using graph Laplacians do not lead to faster minimax rates of convergence. Thus, the estimators that use the unlabeled data do not have smaller risk than the estimators that use only labeled data. We then develop several new approaches that provably lead to improved performance. The statistical tools of minimax analysis are thus used to offer some new perspective on the problem of semisupervised learning. 1
Learning the Structure of Manifolds using Random Projections
 Advances in Neural Information Processing Systems
, 2007
"... We present a simple variant of the kd tree which automatically adapts to intrinsic low dimensional structure in data. 1 ..."
Abstract

Cited by 39 (3 self)
 Add to MetaCart
(Show Context)
We present a simple variant of the kd tree which automatically adapts to intrinsic low dimensional structure in data. 1
Locality sensitive discriminant analysis
 IJCAI
"... Linear Discriminant Analysis (LDA) is a popular dataanalytic tool for studying the class relationship between data points. A major disadvantage of LDA is that it fails to discover the local geometrical structure of the data manifold. In this paper, we introduce a novel linear algorithm for discrimi ..."
Abstract

Cited by 35 (3 self)
 Add to MetaCart
Linear Discriminant Analysis (LDA) is a popular dataanalytic tool for studying the class relationship between data points. A major disadvantage of LDA is that it fails to discover the local geometrical structure of the data manifold. In this paper, we introduce a novel linear algorithm for discriminant analysis, called Locality Sensitive Discriminant Analysis (LSDA). When there is no sufficient training samples, local structure is generally more important than global structure for discriminant analysis. By discovering the local manifold structure, LSDA finds a projection which maximizes the margin between data points from different classes at each local area. Specifically, the data points are mapped into a subspace in which the nearby points with the same label are close to each other while the nearby points with different labels are far apart. Experiments carried out on several standard face databases show a clear improvement over the results of LDAbased recognition.
Semisupervised Hierarchical Models for 3D Human Pose Reconstruction
"... Recent research in visual inference from monocular images has shown that discriminatively trained imagebased predictors can provide fast, automatic qualitative 3D reconstructions of human body pose or scene structure in realworld environments. However, the stability of existing image representation ..."
Abstract

Cited by 34 (4 self)
 Add to MetaCart
(Show Context)
Recent research in visual inference from monocular images has shown that discriminatively trained imagebased predictors can provide fast, automatic qualitative 3D reconstructions of human body pose or scene structure in realworld environments. However, the stability of existing image representations tends to be perturbed by deformations and misalignments in the training set, which, in turn, degrade the quality of learning and generalization. In this paper we advocate the semisupervised learning of hierarchical image descriptions in order to better tolerate variability at multiple levels of detail. We combine multilevel encodings with improved stability to geometric transformations, with metric learning and semisupervised manifold regularization methods in order to further profile them for taskinvariance – resistance to background clutter and within the same human pose class variance. We quantitatively analyze the effectiveness of both descriptors and learning methods and show that each one can contribute, sometimes substantially, to more reliable 3D human pose estimates in cluttered images. 1.