Results 1  10
of
37
Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifolds
 Journal of Machine Learning Research
, 2003
"... The problem of dimensionality reduction arises in many fields of information processing, including machine learning, data compression, scientific visualization, pattern recognition, and neural computation. ..."
Abstract

Cited by 252 (8 self)
 Add to MetaCart
The problem of dimensionality reduction arises in many fields of information processing, including machine learning, data compression, scientific visualization, pattern recognition, and neural computation.
Learning and Design of Principal Curves
, 2000
"... Principal curves have been defined as ``self consistent'' smooth curves which pass through the ``middle'' of a $d$dimensional probability distribution or data cloud. They give a summary of the data and also serve as an efficient feature extraction tool. We take a new approach by defining principal ..."
Abstract

Cited by 74 (5 self)
 Add to MetaCart
Principal curves have been defined as ``self consistent'' smooth curves which pass through the ``middle'' of a $d$dimensional probability distribution or data cloud. They give a summary of the data and also serve as an efficient feature extraction tool. We take a new approach by defining principal curves as continuous curves of a given length which minimize the expected squared distance between the curve and points of the space randomly chosen according to a given distribution. The new definition makes it possible to theoretically analyze principal curve learning from training data and it also leads to a new practical construction. Our theoretical learning scheme chooses a curve from a class of polygonal lines with $k$ segments and with a given total length, to minimize the average squared distance over $n$ training points drawn independently. Convergence properties of this learning scheme are analyzed and a practical version of this theoretical algorithm is implemented. In each iteration of the algorithm a new vertex is added to the polygonal line and the positions of the vertices are updated so that they minimize a penalized squared distance criterion. Simulation results demonstrate that the new algorithm compares favorably with previous methods both in terms of performance and computational complexity, and is more robust to varying data models.
Principal Curves Revisited
 Statistics and Computing
, 1992
"... A principal curve (Hastie and Stuetzle, 1989) is a smooth curve passing through the "middle" of a distribution or data cloud, and is a generalization of linear principal components. We give an alternative definition of a principal curve, based on a mixture model. Estimation is carried out through an ..."
Abstract

Cited by 50 (0 self)
 Add to MetaCart
A principal curve (Hastie and Stuetzle, 1989) is a smooth curve passing through the "middle" of a distribution or data cloud, and is a generalization of linear principal components. We give an alternative definition of a principal curve, based on a mixture model. Estimation is carried out through an EM algorithm. Some comparisons are made to the Hastie Stuetzle definition.
Principal Manifolds and Bayesian Subspaces for Visual Recognition
 International Conference on Computer Vision
, 1999
"... We investigate the use of linear and nonlinear principal manifolds for learning lowdimensional representations for visual recognition. Three techniques: Principal Component Analysis (PCA), Independent Component Analysis (ICA) and Nonlinear PCA (NLPCA) are examined and tested in a visual recognition ..."
Abstract

Cited by 39 (1 self)
 Add to MetaCart
We investigate the use of linear and nonlinear principal manifolds for learning lowdimensional representations for visual recognition. Three techniques: Principal Component Analysis (PCA), Independent Component Analysis (ICA) and Nonlinear PCA (NLPCA) are examined and tested in a visual recognition experiment using a large gallery of facial images from the ¨FERET¨database. We compare the recognition performance of a nearestneighbour matching rule with each principal manifold representation to that of a maximum a posteriori (MAP) matching rule using a Bayesian similarity measure derived from probabilistic subspaces and demonstrate the superiority of the latter.
Piecewise Linear Skeletonization Using Principal Curves
, 2002
"... We propose an algorithm to find piecewise linear skeletons of handwritten characters by using principal curves. The development of the method was inspired by the apparent similarity between the definition of principal curves (smooth curves which pass through the "middle" of a cloud of points) and t ..."
Abstract

Cited by 32 (0 self)
 Add to MetaCart
We propose an algorithm to find piecewise linear skeletons of handwritten characters by using principal curves. The development of the method was inspired by the apparent similarity between the definition of principal curves (smooth curves which pass through the "middle" of a cloud of points) and the medial axis (smooth curves that go equidistantly from the contours of a character image). The central fittingandsmoothing step of the algorithm is an extension of the polygonal line algorithm [1, 2] which approximates principal curves of data sets by piecewise linear curves. The polygonal line algorithm is extended to find principal graphs and complemented with two steps specific to the task of skeletonization: an initialization method to improve the structural quality of the skeleton produced by the initialization method.
Face Recognition in Subspaces
 IN: S.Z. LI, A.K. JAIN (EDS.), HANDBOOK OF FACE RECOGNITION
, 2004
"... Images of faces, represented as highdimensional pixel arrays, often belong to a manifold of intrinsically low dimension. Face recognition, and computer vision research in general, has witnessed a growing interest in techniques that capitalize on this observation, and apply algebraic and statisti ..."
Abstract

Cited by 26 (0 self)
 Add to MetaCart
Images of faces, represented as highdimensional pixel arrays, often belong to a manifold of intrinsically low dimension. Face recognition, and computer vision research in general, has witnessed a growing interest in techniques that capitalize on this observation, and apply algebraic and statistical tools for extraction and analysis of the underlying manifold. In this chapter we describe in roughly chronological order techniques that identify, parameterize and analyze linear and nonlinear subspaces, from the original Eigenfaces technique to the recently introduced Bayesian method for probabilistic similarity analysis, and discuss comparative experimental evaluation of some of these techniques. We also discuss practical issues related to the application of subspace methods for varying pose, illumination and expression.
Nonlinear Partial Least Squares
, 1995
"... We propose a new nonparametric regression method for highdimensional data, nonlinear partial least squares (NLPLS). NLPLS is motivated by projectionbased regression methods, e.g., partial least squares (PLS), projection pursuit (PPR), and feedforward neural networks. The model takes the form of a ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
We propose a new nonparametric regression method for highdimensional data, nonlinear partial least squares (NLPLS). NLPLS is motivated by projectionbased regression methods, e.g., partial least squares (PLS), projection pursuit (PPR), and feedforward neural networks. The model takes the form of a composition of two functions. The first function in the composition projects the predictor variables onto a lowerdimensional curve or surface yielding scores, and the second predicts the response variable from the scores. We implement NLPLS with feedforward neural networks. NLPLS will often produce a more parsimonious model (fewer score vectors) than projectionbased methods, and the model is well suited for detecting outliers and future covariates requiring extrapolation. The scores are also shown to have useful interpretations. We also extend the model for multiple response variables and discuss situations when multiple response variab...
Principal surfaces from unsupervised kernel regression
 IEEE Transactions on Pattern Analysis and Machine Intelligence
, 2005
"... Abstract—We propose a nonparametric approach to learning of principal surfaces based on an unsupervised formulation of the NadarayaWatson kernel regression estimator. As compared with previous approaches to principal curves and surfaces, the new method offers several advantages: First, it provides ..."
Abstract

Cited by 16 (9 self)
 Add to MetaCart
Abstract—We propose a nonparametric approach to learning of principal surfaces based on an unsupervised formulation of the NadarayaWatson kernel regression estimator. As compared with previous approaches to principal curves and surfaces, the new method offers several advantages: First, it provides a practical solution to the model selection problem because all parameters can be estimated by leaveoneout crossvalidation without additional computational cost. In addition, our approach allows for a convenient incorporation of nonlinear spectral methods for parameter initialization, beyond classical initializations based on linear PCA. Furthermore, it shows a simple way to fit principal surfaces in general feature spaces, beyond the usual data space setup. The experimental results illustrate these convenient features on simulated and real data. Index Terms—Dimensionality reduction, principal curves, principal surfaces, density estimation, model selection, kernel methods. æ 1
Principal Curves: Learning, Design, And Applications
, 1999
"... The subjects of this thesis are unsupervised learning in general, and principal curves in particular. Principal curves were originally defined by Hastie \cite{Has84} and Hastie and Stuetzle \cite{HaSt89} (hereafter HS) to formally capture the notion of a smooth curve passing through the ``middle'' o ..."
Abstract

Cited by 14 (3 self)
 Add to MetaCart
The subjects of this thesis are unsupervised learning in general, and principal curves in particular. Principal curves were originally defined by Hastie \cite{Has84} and Hastie and Stuetzle \cite{HaSt89} (hereafter HS) to formally capture the notion of a smooth curve passing through the ``middle'' of a $d$dimensional probability distribution or data cloud. Based on the definition, HS also developed an algorithm for constructing principal curves of distributions and data sets. The field has been very active since Hastie and Stuetzle's groundbreaking work. Numerous alternative definitions and methods for estimating principal curves have been proposed, and principal curves were further analyzed and compared with other unsupervised learning techniques. Several applications in various areas including image analysis, feature extraction, and speech processing demonstrated that principal curves are not only of theoretical interest, but they also have a legitimate place in the family of practical unsupervised learning techniques. Although the concept of principal curves as considered by HS has several appealing characteristics, complete theoretical analysis of the model seems to be rather hard. This motivated us to redefine principal curves in a manner that allowed us to carry out extensive theoretical analysis while preserving the informal notion of principal curves. Our first contribution to the area is, hence, a new {\em theoretical model} that is analyzed by using tools of statistical learning theory. Our main result here is the first known consistency proof of a principal curve estimation scheme. The theoretical model proved to be too restrictive to be practical. However, it inspired the design of a new {\em practical algorithm} to estimate principal curves based on data. The polygonal line algorithm, which compares favorably with previous methods both in terms of performance and computational complexity, is our second contribution to the area of principal curves. To complete the picture, in the last part of the thesis we consider an {\em application} of the polygonal line algorithm to handwritten character skeletonization.
A Polygonal Line Algorithm for Constructing Principal Curves
, 1999
"... Principal curves have been defined as ``self consistent'' smooth curves which pass through the ``middle'' of a $d$dimensional probability distribution or data cloud. Recently, we \cite{KeKrLiZe98a} have offered a new approach by defining principal curves as continuous curves of a given length which ..."
Abstract

Cited by 13 (2 self)
 Add to MetaCart
Principal curves have been defined as ``self consistent'' smooth curves which pass through the ``middle'' of a $d$dimensional probability distribution or data cloud. Recently, we \cite{KeKrLiZe98a} have offered a new approach by defining principal curves as continuous curves of a given length which minimize the expected squared distance between the curve and points of the space randomly chosen according to a given distribution. The new definition made it possible to carry out a theoretical analysis of learning principal curves from training data. In this paper we propose a practical construction based on the new definition. Simulation results demonstrate that the new algorithm compares favorably with previous methods both in terms of performance and computational complexity.