Results 1  10
of
124
Geometric mean for subspace selection
 TIANJIN UNIVERSITY. Downloaded on December 8, 2009 at 04:33 from IEEE Xplore. Restrictions apply. YUAN et al.: BINARY SPARSE NONNEGATIVE MATRIX FACTORIZATION 777
, 2009
"... Abstract—Subspace selection approaches are powerful tools in pattern classification and data visualization. One of the most important subspace approaches is the linear dimensionality reduction step in the Fisher’s linear discriminant analysis (FLDA), which has been successfully employed in many fiel ..."
Abstract

Cited by 52 (11 self)
 Add to MetaCart
(Show Context)
Abstract—Subspace selection approaches are powerful tools in pattern classification and data visualization. One of the most important subspace approaches is the linear dimensionality reduction step in the Fisher’s linear discriminant analysis (FLDA), which has been successfully employed in many fields such as biometrics, bioinformatics, and multimedia information management. However, the linear dimensionality reduction step in FLDA has a critical drawback: for a classification task with c classes, if the dimension of the projected subspace is strictly lower than c 1, the projection to a subspace tends to merge those classes, which are close together in the original feature space. If separate classes are sampled from Gaussian distributions, all with identical covariance matrices, then the linear dimensionality reduction step in FLDA maximizes the mean value of the KullbackLeibler (KL) divergences between different classes. Based on this viewpoint, the geometric mean for subspace selection is studied in this paper. Three criteria are analyzed: 1) maximization of the geometric mean of the KL divergences, 2) maximization of the geometric mean of the normalized KL divergences, and 3) the combination of 1 and 2. Preliminary experimental results based on synthetic data, UCI Machine Learning Repository, and handwriting digits show that the third criterion is a potential discriminative subspace selection method, which significantly reduces the class separation problem in comparing with the linear dimensionality reduction step in FLDA and its several representative extensions. Index Terms—Arithmetic mean, Fisher’s linear discriminant analysis (FLDA), geometric mean, KullbackLeibler (KL) divergence, machine learning, subspace selection (or dimensionality reduction), visualization. Ç 1
Changepoint detection in timeseries data by direct densityratio estimation
 Proceedings of 2009 SIAM International Conference on Data Mining (SDM2009
, 2009
"... Changepoint detection is the problem of discovering time points at which properties of timeseries data change. This covers a broad range of realworld problems and has been actively discussed in the community of statistics and data mining. In this paper, we present a novel nonparametric approach ..."
Abstract

Cited by 25 (5 self)
 Add to MetaCart
(Show Context)
Changepoint detection is the problem of discovering time points at which properties of timeseries data change. This covers a broad range of realworld problems and has been actively discussed in the community of statistics and data mining. In this paper, we present a novel nonparametric approach to detecting the change of probability distributions of sequence data. Our key idea is to estimate the ratio of probability densities, not the probability densities themselves. This formulation allows us to avoid nonparametric density estimation, which is known to be a difficult problem. We provide a changepoint detection algorithm based on direct densityratio estimation that can be computed very efficiently in an online manner. The usefulness of the proposed method is demonstrated through experiments using artificial and real datasets.
SemiSupervised Local Fisher Discriminant Analysis for Dimensionality Reduction
 PAKDD
, 2008
"... When only a small number of labeled samples are available, supervised dimensionality reduction methods tend to perform poorly due to overfitting. In such cases, unlabeled samples could be useful in improving the performance. In this paper, we propose a semisupervised dimensionality reduction method ..."
Abstract

Cited by 24 (4 self)
 Add to MetaCart
(Show Context)
When only a small number of labeled samples are available, supervised dimensionality reduction methods tend to perform poorly due to overfitting. In such cases, unlabeled samples could be useful in improving the performance. In this paper, we propose a semisupervised dimensionality reduction method which preserves the global structure of unlabeled samples in addition to separating labeled samples in different classes from each other. The proposed method has an analytic form of the globally optimal solution and it can be computed based on eigendecompositions. Therefore, the proposed method is computationally reliable and efficient. We show the effectiveness of the proposed method through extensive simulations with benchmark data sets.
Direct Densityratio Estimation with Dimensionality Reduction via Leastsquares Heterodistributional Subspace Search
 NEURAL NETWORKS, VOL.24, NO.2, PP.183–198
, 2011
"... Methods for directly estimating the ratio of two probability density functions have been actively explored recently since they can be used for various data processing tasks such as nonstationarity adaptation, outlier detection, and feature selection. In this paper, we develop a new method which inc ..."
Abstract

Cited by 23 (15 self)
 Add to MetaCart
(Show Context)
Methods for directly estimating the ratio of two probability density functions have been actively explored recently since they can be used for various data processing tasks such as nonstationarity adaptation, outlier detection, and feature selection. In this paper, we develop a new method which incorporates dimensionality reduction into a direct densityratio estimation procedure. Our key idea is to find a lowdimensional subspace in which densities are significantly different and perform density ratio estimation only in this subspace. The proposed method, D³LHSS (Direct Densityratio estimation with Dimensionality reduction via Leastsquares Heterodistributional Subspace Search), is shown to overcome the limitation of baseline methods.
Localitypreserving dimensionality reduction and classification for hyperspectral image analysis
 IEEE Transactions on Geoscience and Remote Sensing
, 2012
"... Abstract—Hyperspectral imagery typically provides a wealth of information captured in a wide range of the electromagnetic spectrum for each pixel in the image; however, when used in statistical patternclassification tasks, the resulting highdimensional feature spaces often tend to result in illc ..."
Abstract

Cited by 18 (9 self)
 Add to MetaCart
(Show Context)
Abstract—Hyperspectral imagery typically provides a wealth of information captured in a wide range of the electromagnetic spectrum for each pixel in the image; however, when used in statistical patternclassification tasks, the resulting highdimensional feature spaces often tend to result in illconditioned formulations. Popular dimensionalityreduction techniques such as principal component analysis, linear discriminant analysis, and their variants typically assume a Gaussian distribution. The quadratic maximumlikelihood classifier commonly employed for hyperspectral analysis also assumes singleGaussian classconditional distributions. Departing from this singleGaussian assumption, a classification paradigm designed to exploit the rich statistical structure of the data is proposed. The proposed framework employs local Fisher’s discriminant analysis to reduce the dimensionality of the data while preserving its multimodal structure, while a subsequent Gaussian mixture model or support vector machine provides effective classification of the reduceddimension multimodal data. Experimental results on several different multipleclass hyperspectralclassification tasks demonstrate that the proposed approach significantly outperforms several traditional alternatives. Index Terms—Dimensionality reduction, Gaussianmixturemodel (GMM), hyperspectral data, local discriminant analysis, support vector machine. I.
Human action recognition using local spatiotemporal discriminant embedding
 in Proc. CVPR
, 2008
"... Human action video sequences can be considered as nonlinear dynamic shape manifolds in the space of image frames. In this paper, we address learning and classifying human actions on embedded lowdimensional manifolds. We propose a novel manifold embedding method, called Local SpatioTemporal Discrim ..."
Abstract

Cited by 15 (0 self)
 Add to MetaCart
(Show Context)
Human action video sequences can be considered as nonlinear dynamic shape manifolds in the space of image frames. In this paper, we address learning and classifying human actions on embedded lowdimensional manifolds. We propose a novel manifold embedding method, called Local SpatioTemporal Discriminant Embedding (LSTDE). The discriminating capabilities of the proposed method are twofold: (1) for local spatial discrimination, LSTDE projects data points (silhouettebased image frames of human action sequences) in a local neighborhood into the embedding space where data points of the same action class are close while those of different classes are far apart; (2) in such a local neighborhood, each data point has an associated short video segment, which forms a local temporal subspace on the embedded manifold. LSTDE finds an optimal embedding which maximizes the principal angles between those temporal subspaces associated with data points of different classes. Benefiting from the joint spatiotemporal discriminant embedding, our method is potentially more powerful for classifying human actions with similar spacetime shapes, and is able to perform recognition on a framebyframe or short video segment basis. Experimental results demonstrate that our method can accurately recognize human actions, and can improve the recognition performance over some representative manifold embedding methods, especially on highly confusing human action types. 1.
On InformationMaximization Clustering: Tuning Parameter Selection and Analytic Solution
"... Informationmaximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it only involves continuous optimization of model parameters, which is ..."
Abstract

Cited by 13 (8 self)
 Add to MetaCart
Informationmaximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it only involves continuous optimization of model parameters, which is substantially easier to solve than discrete optimization of cluster assignments. However, existing methods still involve nonconvex optimization problems, and therefore finding a good local optimal solution is not straightforward in practice. In this paper, we propose an alternative informationmaximization clustering method based on a squaredloss variant of mutual information. This novel approach gives a clustering solution analytically in a computationally efficient way via kernel eigenvalue decomposition. Furthermore, we provide a practical model selection procedure that allows us to objectively optimize tuning parameters included in the kernel function. Through experiments, we demonstrate the usefulness of the proposed approach. 1.
Nearest Regularized Subspace for Hyperspectral Classification
 IEEE Transactions on Geoscience and Remote Sensing, Submitted April 2012. Revised
, 2012
"... Abstract—A classifier that couples nearestsubspace classification with a distanceweighted Tikhonov regularization is proposed for hyperspectral imagery. The resulting nearestregularizedsubspace classifier seeks an approximation of each testing sample via a linear combination of training samples w ..."
Abstract

Cited by 11 (10 self)
 Add to MetaCart
(Show Context)
Abstract—A classifier that couples nearestsubspace classification with a distanceweighted Tikhonov regularization is proposed for hyperspectral imagery. The resulting nearestregularizedsubspace classifier seeks an approximation of each testing sample via a linear combination of training samples within each class. The class label is then derived according to the class which best approximates the test sample. The distanceweighted Tikhonov regularization is then modified by measuring distance within a localitypreserving lowerdimensional subspace. Furthermore, a competitive process among the classes is proposed to simplify parameter tuning. Classification results for several hyperspectral image data sets demonstrate superior performance of the proposed approach when compared to other, more traditional classification techniques. Index Terms—Classification, hyperspectral data, Tikhonov regularization. I.
Localitypreserving discriminant analysis in kernelinduced feature spaces for hyperspectral image classification
 IEEE Geoscience and Remote Sensing Letters
, 2011
"... Localitypreserving projection as well as local Fisher discriminant analysis is applied for dimensionality reduction of hyperspectral imagery based on both spatial and spectral information. These techniques preserve the local geometric structure of hyperspectral data into a lowdimensional subspace ..."
Abstract

Cited by 10 (8 self)
 Add to MetaCart
(Show Context)
Localitypreserving projection as well as local Fisher discriminant analysis is applied for dimensionality reduction of hyperspectral imagery based on both spatial and spectral information. These techniques preserve the local geometric structure of hyperspectral data into a lowdimensional subspace wherein a Gaussianmixturemodel classifier is then considered. In the proposed classification system, local spatial information—which is expected to be more multimodal than strictly spectral features—is used. Results with experimental hyperspectral data demonstrate that this system outperforms traditional classification approaches. Index Terms — Dimensionality reduction, linear discriminant analysis, hyperspectral data, pattern classification. 1.
Gsml: A unified framework for sparse metric learning
 In Data Mining, 2009. ICDM’09. Ninth IEEE International Conference on
, 2009
"... There has been significant recent interest in sparse metric learning (SML) in which we simultaneously learn both a good distance metric and a lowdimensional representation. Unfortunately, the performance of existing sparse metric learning approaches is usually limited because the authors assumed ce ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
(Show Context)
There has been significant recent interest in sparse metric learning (SML) in which we simultaneously learn both a good distance metric and a lowdimensional representation. Unfortunately, the performance of existing sparse metric learning approaches is usually limited because the authors assumed certain problem relaxations or they target the SML objective indirectly. In this paper, we propose a Generalized Sparse Metric Learning method (GSML). This novel framework offers a unified view for understanding many of the popular sparse metric learning algorithms including the Sparse Metric Learning framework proposed in [15], the Large Margin Nearest Neighbor (LMNN) [21][22], and the Dranking Vector Machine (Dranking VM) [14]. Moreover, GSML also establishes a close relationship with the