Results 1  10
of
53
Nonlinear component analysis as a kernel eigenvalue problem

, 1996
"... We describe a new method for performing a nonlinear form of Principal Component Analysis. By the use of integral operator kernel functions, we can efficiently compute principal components in highdimensional feature spaces, related to input space by some nonlinear map; for instance the space of all ..."
Abstract

Cited by 1425 (79 self)
 Add to MetaCart
(Show Context)
We describe a new method for performing a nonlinear form of Principal Component Analysis. By the use of integral operator kernel functions, we can efficiently compute principal components in highdimensional feature spaces, related to input space by some nonlinear map; for instance the space of all possible 5pixel products in 16x16 images. We give the derivation of the method, along with a discussion of other techniques which can be made nonlinear with the kernel approach; and present first experimental results on nonlinear feature extraction for pattern recognition.
Efficient BackProp
, 1998
"... . The convergence of backpropagation learning is analyzed so as to explain common phenomenon observed by practitioners. Many undesirable behaviors of backprop can be avoided with tricks that are rarely exposed in serious technical publications. This paper gives some of those tricks, and offers expl ..."
Abstract

Cited by 192 (29 self)
 Add to MetaCart
. The convergence of backpropagation learning is analyzed so as to explain common phenomenon observed by practitioners. Many undesirable behaviors of backprop can be avoided with tricks that are rarely exposed in serious technical publications. This paper gives some of those tricks, and offers explanations of why they work. Many authors have suggested that secondorder optimization methods are advantageous for neural net training. It is shown that most "classical" secondorder methods are impractical for large neural networks. A few methods are proposed that do not have these limitations. 1 Introduction Backpropagation is a very popular neural network learning algorithm because it is conceptually simple, computationally efficient, and because it often works. However, getting it to work well, and sometimes to work at all, can seem more of an art than a science. Designing and training a network using backprop requires making many seemingly arbitrary choices such as the number ...
Kernel PCA and DeNoising in Feature Spaces
 ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 11
, 1999
"... Kernel PCA as a nonlinear feature extractor has proven powerful as a preprocessing step for classification algorithms. But it can also be considered as a natural generalization of linear principal component analysis. This gives rise to the question how to use nonlinear features for data compress ..."
Abstract

Cited by 156 (15 self)
 Add to MetaCart
Kernel PCA as a nonlinear feature extractor has proven powerful as a preprocessing step for classification algorithms. But it can also be considered as a natural generalization of linear principal component analysis. This gives rise to the question how to use nonlinear features for data compression, reconstruction, and denoising, applications common in linear PCA. This is a nontrivial task, as the results provided by kernel PCA live in some high dimensional feature space and need not have preimages in input space. This work presents ideas for finding approximate preimages, focusing on Gaussian kernels, and shows experimental results using these preimages in data reconstruction and denoising on toy examples as well as on real world data.
Constructing Descriptive and Discriminative Nonlinear Features: Rayleigh Coefficients in Kernel Feature Spaces
, 2003
"... We incorporate prior knowledge to construct nonlinear algorithms for invariant feature extraction and discrimination. Employing a unified framework in terms of a nonlinearized variant of the Rayleigh coefficient, we propose nonlinear generalizations of Fisher's discriminant and oriented PCA usi ..."
Abstract

Cited by 67 (5 self)
 Add to MetaCart
We incorporate prior knowledge to construct nonlinear algorithms for invariant feature extraction and discrimination. Employing a unified framework in terms of a nonlinearized variant of the Rayleigh coefficient, we propose nonlinear generalizations of Fisher's discriminant and oriented PCA using support vector kernel functions. Extensive simulations show the utility of our approach.
Invariant Feature Extraction and Classification in Kernel Spaces
"... We incorporate prior knowledge to construct nonlinear algorithms for invariant feature extraction and discrimination. Employing a unified framework in terms of a nonlinear variant of the Rayleigh coefficient, we propose nonlinear generalizations of Fisher's discriminant and oriented PCA us ..."
Abstract

Cited by 52 (7 self)
 Add to MetaCart
We incorporate prior knowledge to construct nonlinear algorithms for invariant feature extraction and discrimination. Employing a unified framework in terms of a nonlinear variant of the Rayleigh coefficient, we propose nonlinear generalizations of Fisher's discriminant and oriented PCA using Support Vector kernel functions.
Kernel PCA Pattern Reconstruction via Approximate PreImages
, 1998
"... Algorithms based on Mercer kernels construct their solutions in terms of expansions in a highdimensional feature space F . Previous work has shown that all algorithms which can be formulated in terms of dot products in F can be performed using a kernel without explicitly working in F . The list of ..."
Abstract

Cited by 43 (3 self)
 Add to MetaCart
Algorithms based on Mercer kernels construct their solutions in terms of expansions in a highdimensional feature space F . Previous work has shown that all algorithms which can be formulated in terms of dot products in F can be performed using a kernel without explicitly working in F . The list of such algorithms includes support vector machines and nonlinear kernel principal component extraction. So far, however, it did not include the reconstruction of patterns from their largest nonlinear principal components, a technique which is common practice in linear principal component analysis. The present work proposes an idea for approximately performing this task. As an illustrative example, an application to the denoising of data clusters is presented. 1 Kernels and Feature Spaces A Mercer kernel is a function k(x; y) which for all data sets fx 1 ; : : : ; x ` g ae R N gives rise to a positive (not necessarily definite) matrix K ij := k(x i ; x j ) [4]. One can show that using k ins...
Geometric Methods for Feature Extraction and Dimensional Reduction
 In L. Rokach and O. Maimon (Eds.), Data
, 2005
"... Abstract We give a tutorial overview of several geometric methods for feature extraction and dimensional reduction. We divide the methods into projective methods and methods that model the manifold on which the data lies. For projective methods, we review projection pursuit, principal component anal ..."
Abstract

Cited by 42 (1 self)
 Add to MetaCart
(Show Context)
Abstract We give a tutorial overview of several geometric methods for feature extraction and dimensional reduction. We divide the methods into projective methods and methods that model the manifold on which the data lies. For projective methods, we review projection pursuit, principal component analysis (PCA), kernel PCA, probabilistic PCA, and oriented PCA; and for the manifold methods, we review multidimensional scaling (MDS), landmark MDS, Isomap, locally linear embedding, Laplacian eigenmaps and spectral clustering. The Nyström method, which links several of the algorithms, is also reviewed. The goal is to provide a selfcontained review of the concepts and mathematics underlying these algorithms.
Fast active appearance model search using canonical correlation analysis
"... Abstract—A fast AAM search algorithm based on canonical correlation analysis (CCAAAM) is introduced. It efficiently models the dependency between texture residuals and model parameters during search. Experiments show that CCAAAMs, while requiring similar implementation effort, consistently outperf ..."
Abstract

Cited by 33 (3 self)
 Add to MetaCart
(Show Context)
Abstract—A fast AAM search algorithm based on canonical correlation analysis (CCAAAM) is introduced. It efficiently models the dependency between texture residuals and model parameters during search. Experiments show that CCAAAMs, while requiring similar implementation effort, consistently outperform standard search with regard to convergence speed by a factor of four. Index Terms—Image processing and computer vision, active appearance models, statistical image models, subspace methods, medical imaging. Ç 1
A robust PCA algorithm for building representations from panoramic images
 In European Conference Computer Vision
, 2002
"... Abstract. Appearancebased modeling of objects and scenes using PCA has been successfully applied in many recognition tasks. Robust methods which have made the recognition stage less susceptible to outliers, occlusions, and varying illumination have further enlarged the domain of applicability. Howe ..."
Abstract

Cited by 30 (10 self)
 Add to MetaCart
(Show Context)
Abstract. Appearancebased modeling of objects and scenes using PCA has been successfully applied in many recognition tasks. Robust methods which have made the recognition stage less susceptible to outliers, occlusions, and varying illumination have further enlarged the domain of applicability. However, much less research has been done in achieving robustness in the learning stage. In this paper, we propose a novel robust PCA method for obtaining a consistent subspace representation in the presence of outlying pixels in the training images. The method is based on the EM algorithm for estimation of principal subspaces in the presence of missing data. By treating the outlying points as missing pixels, we arrive at a robust PCA representation. We demonstrate experimentally that the proposed method is efficient. In addition, we apply the method to a set of panoramic images to build a representation that enables surveillance and viewbased mobile robot localization. 1
Nonlinear PCA: a missing data approach
 BIOINFORMATICS
, 2005
"... Motivation: Visualising and analysing the potential nonlinear structure of a data set is becoming an important task in molecular biology. This is even more challenging when the data have missing values. Results: Here, we propose an inverse model that performs nonlinear principal component analysis ( ..."
Abstract

Cited by 27 (7 self)
 Add to MetaCart
Motivation: Visualising and analysing the potential nonlinear structure of a data set is becoming an important task in molecular biology. This is even more challenging when the data have missing values. Results: Here, we propose an inverse model that performs nonlinear principal component analysis (NLPCA) from incomplete data sets. Missing values are ignored while optimising the model, but can be estimated afterwards. Results are shown for both artificial and experimental data sets. In contrast to linear methods, nonlinear methods were able to give better missing value estimations for nonlinear structured data. Application: We applied this technique to a time course of metabolite data from a cold stress experiment on the model plant Arabidopsis thaliana, and could approximate the mapping function from any time point to the metabolite responses. Thus, the inverse NLPCA provides greatly improved information for better understanding the complex response to cold stress.