Results 11  20
of
297
Subclass discriminant analysis
 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
, 2006
"... Over the years, many Discriminant Analysis (DA) algorithms have been proposed for the study of highdimensional data in a large variety of problems. Each of these algorithms is tuned to a specific type of data distribution (that which best models the problem at hand). Unfortunately, in most problem ..."
Abstract

Cited by 60 (10 self)
 Add to MetaCart
Over the years, many Discriminant Analysis (DA) algorithms have been proposed for the study of highdimensional data in a large variety of problems. Each of these algorithms is tuned to a specific type of data distribution (that which best models the problem at hand). Unfortunately, in most problems the form of each class pdf is a priori unknown, and the selection of the DA algorithm that best fits our data is done over trialanderror. Ideally, one would like to have a single formulation which can be used for most distribution types. This can be achieved by approximating the underlying distribution of each class with a mixture of Gaussians. In this approach, the major problem to be addressed is that of determining the optimal number of Gaussians per class, i.e., the number of subclasses. In this paper, two criteria able to find the most convenient division of each class into a set of subclasses are derived. Extensive experimental results are shown using five databases. Comparisons are given against Linear Discriminant Analysis (LDA), Direct LDA (DLDA), Heteroscedastic LDA (HLDA), Nonparametric DA (NDA), and KernelBased LDA (KLDA). We show that our method is always the best or comparable to the best.
Graph embedding: a general framework for dimensionality reduction. CVPR
, 2005
"... In the last decades, a large family of algorithmsũ supervised or unsupervised; stemming from statistic or geometry theoryũhave been proposed to provide different solutions to the problem of dimensionality reduction. In this paper, beyond the different motivations of these algorithms, we propose a ..."
Abstract

Cited by 57 (8 self)
 Add to MetaCart
(Show Context)
In the last decades, a large family of algorithmsũ supervised or unsupervised; stemming from statistic or geometry theoryũhave been proposed to provide different solutions to the problem of dimensionality reduction. In this paper, beyond the different motivations of these algorithms, we propose a general framework, graph embedding along with its linearization and kernelization, which in theory reveals the underlying objective shared by most previous algorithms. It presents a unified perspective to understand these algorithms; that is, each algorithm can be considered as the direct graph embedding or its linear/kernel extension of some specific graph characterizing certain statistic or geometry property of a data set. Furthermore, this framework is a general platform to develop new algorithm for dimensionality reduction. To this end, we propose a new supervised algorithm, Marginal Fisher Analysis (MFA), for dimensionality reduction by designing two graphs that characterize the intraclass compactness and interclass separability, respectively. MFA measures the intraclass compactness with the distance between each data point and its neighboring points of the same class, and measures the interclass separability with the class margins; thus it overcomes the limitations of traditional Linear Discriminant Analysis algorithm in terms of data distribution assumptions and available projection directions. The toy problem on artificial data and the real face recognition experiments both show the superiority of our proposed MFA in comparison to LDA. 1.
Geometric mean for subspace selection
 TIANJIN UNIVERSITY. Downloaded on December 8, 2009 at 04:33 from IEEE Xplore. Restrictions apply. YUAN et al.: BINARY SPARSE NONNEGATIVE MATRIX FACTORIZATION 777
, 2009
"... Abstract—Subspace selection approaches are powerful tools in pattern classification and data visualization. One of the most important subspace approaches is the linear dimensionality reduction step in the Fisher’s linear discriminant analysis (FLDA), which has been successfully employed in many fiel ..."
Abstract

Cited by 52 (11 self)
 Add to MetaCart
(Show Context)
Abstract—Subspace selection approaches are powerful tools in pattern classification and data visualization. One of the most important subspace approaches is the linear dimensionality reduction step in the Fisher’s linear discriminant analysis (FLDA), which has been successfully employed in many fields such as biometrics, bioinformatics, and multimedia information management. However, the linear dimensionality reduction step in FLDA has a critical drawback: for a classification task with c classes, if the dimension of the projected subspace is strictly lower than c 1, the projection to a subspace tends to merge those classes, which are close together in the original feature space. If separate classes are sampled from Gaussian distributions, all with identical covariance matrices, then the linear dimensionality reduction step in FLDA maximizes the mean value of the KullbackLeibler (KL) divergences between different classes. Based on this viewpoint, the geometric mean for subspace selection is studied in this paper. Three criteria are analyzed: 1) maximization of the geometric mean of the KL divergences, 2) maximization of the geometric mean of the normalized KL divergences, and 3) the combination of 1 and 2. Preliminary experimental results based on synthetic data, UCI Machine Learning Repository, and handwriting digits show that the third criterion is a potential discriminative subspace selection method, which significantly reduces the class separation problem in comparing with the linear dimensionality reduction step in FLDA and its several representative extensions. Index Terms—Arithmetic mean, Fisher’s linear discriminant analysis (FLDA), geometric mean, KullbackLeibler (KL) divergence, machine learning, subspace selection (or dimensionality reduction), visualization. Ç 1
A twostage linear discriminant analysis via QRdecomposition
 IEEE TRANSACTION ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
, 2005
"... Linear Discriminant Analysis (LDA) is a wellknown method for feature extraction and dimension reduction. It has been used widely in many applications involving highdimensional data, such as image and text classification. An intrinsic limitation of classical LDA is the socalled singularity proble ..."
Abstract

Cited by 48 (0 self)
 Add to MetaCart
Linear Discriminant Analysis (LDA) is a wellknown method for feature extraction and dimension reduction. It has been used widely in many applications involving highdimensional data, such as image and text classification. An intrinsic limitation of classical LDA is the socalled singularity problems; that is, it fails when all scatter matrices are singular. Many LDA extensions were proposed in the past to overcome the singularity problems. Among these extensions, PCA+LDA, a twostage method, received relatively more attention. In PCA+LDA, the LDA stage is preceded by an intermediate dimension reduction stage using Principal Component Analysis (PCA). Most previous LDA extensions are computationally expensive, and not scalable, due to the use of Singular Value Decomposition or Generalized Singular Value Decomposition. In this paper, we propose a twostage LDA method, namely LDA/QR, which aims to overcome the singularity problems of classical LDA, while achieving efficiency and scalability simultaneously. The key difference between LDA/QR and PCA+LDA lies in the first stage, where LDA/QR applies QR decomposition to a small matrix involving the class centroids, while PCA+LDA applies PCA to the total scatter matrix involving all training data points. We further justify the proposed algorithm by showing the relationship among LDA/QR and previous LDA methods. Extensive experiments on face images and text documents are presented to show the effectiveness of the proposed algorithm.
Where are linear feature extraction methods applicable
 IEEE Trans. Pattern Anal. Mach. Intell
, 2005
"... Abstract—A fundamental problem in computer vision and pattern recognition is to determine where and, most importantly, why a given technique is applicable. This is not only necessary because it helps us decide which techniques to apply at each given time. Knowing why current algorithms cannot be app ..."
Abstract

Cited by 47 (15 self)
 Add to MetaCart
(Show Context)
Abstract—A fundamental problem in computer vision and pattern recognition is to determine where and, most importantly, why a given technique is applicable. This is not only necessary because it helps us decide which techniques to apply at each given time. Knowing why current algorithms cannot be applied facilitates the design of new algorithms robust to such problems. In this paper, we report on a theoretical study that demonstrates where and why generalized eigenbased linear equations do not work. In particular, we show that when the smallest angle between the ith eigenvector given by the metric to be maximized and the first i eigenvectors given by the metric to be minimized is close to zero, our results are not guaranteed to be correct. Several properties of such models are also presented. For illustration, we concentrate on the classical applications of classification and feature extraction. We also show how we can use our findings to design more robust algorithms. We conclude with a discussion on the broader impacts of our results. Index Terms—Feature extraction, generalized eigenvalue decomposition, performance evaluation, classifiers, pattern recognition. æ 1
A face and palmprint recognition approach based on discriminant DCT feature extraction
 12 Computational Intelligence and Neuroscience
, 2004
"... Abstract—In the field of image processing and recognition, discrete cosine transform (DCT) and linear discrimination are two widely used techniques. Based on them, we present a new face and palmprint recognition approach in this paper. It first uses a twodimensional separability judgment to select ..."
Abstract

Cited by 42 (2 self)
 Add to MetaCart
(Show Context)
Abstract—In the field of image processing and recognition, discrete cosine transform (DCT) and linear discrimination are two widely used techniques. Based on them, we present a new face and palmprint recognition approach in this paper. It first uses a twodimensional separability judgment to select the DCT frequency bands with favorable linear separability. Then from the selected bands, it extracts the linear discriminative features by an improved Fisherface method and performs the classification by the nearest neighbor classifier. We detailedly analyze theoretical advantages of our approach in feature extraction. The experiments on face databases and palmprint database demonstrate that compared to the stateoftheart linear discrimination methods, our approach obtains better classification performance. It can significantly improve the recognition rates for face and palmprint data and effectively reduce the dimension of feature space. Index Terms—Discrete cosine transform (DCT), DCT frequency band selection, improved Fisherface method, linear discrimination technique, twodimensional (2D) separability judgment. I.
Gabor wavelets and General Discriminant Analysis for face identification and verification
, 2007
"... ..."
Ensemblebased discriminant learning with boosting for face recognition
 10, 2009 DRAFT IVP/713183.V2 23
, 2006
"... Abstract—In this paper, we propose a novel ensemblebased approach to boost performance of traditional Linear Discriminant Analysis (LDA)based methods used in face recognition. The ensemblebased approach is based on the recently emerged technique known as “boosting. ” However, it is generally beli ..."
Abstract

Cited by 32 (6 self)
 Add to MetaCart
(Show Context)
Abstract—In this paper, we propose a novel ensemblebased approach to boost performance of traditional Linear Discriminant Analysis (LDA)based methods used in face recognition. The ensemblebased approach is based on the recently emerged technique known as “boosting. ” However, it is generally believed that boostinglike learning rules are not suited to a strong and stable learner such as LDA. To break the limitation, a novel weakness analysis theory is developed here. The theory attempts to boost a strong learner by increasing the diversity between the classifiers created by the learner, at the expense of decreasing their margins, so as to achieve a tradeoff suggested by recent boosting studies for a low generalization error. In addition, a novel distribution accounting for the pairwise class discriminant information is introduced for effective interaction between the booster and the LDAbased learner. The integration of all these methodologies proposed here leads to the novel ensemblebased discriminant learning approach, capable of taking advantage of both the boosting and LDA techniques. Promising experimental results obtained on various difficult face recognition scenarios demonstrate the effectiveness of the proposed approach. We believe that this work is especially beneficial in extending the boosting framework to accommodate general (strong/weak) learners. Index Terms—Boosting, face recognition (FR), linear discriminant analysis, machine learning, mixture of linear models, smallsamplesize (SSS) problem, strong learner.
A Maximum Uncertainty LDAbased approach for Limited Sample Size Problems  with . . .
 IN PROC. MICCAI’04, VOL. LNCS 3216
, 2004
"... A critical issue of applying Linear Discriminant Analysis (LDA) is both the singularity and instability of the withinclass scatter matrix. In practice, particularly in image recognition applications such as face recognition, there are often a large number of pixels or preprocessed features availab ..."
Abstract

Cited by 25 (7 self)
 Add to MetaCart
(Show Context)
A critical issue of applying Linear Discriminant Analysis (LDA) is both the singularity and instability of the withinclass scatter matrix. In practice, particularly in image recognition applications such as face recognition, there are often a large number of pixels or preprocessed features available, but the total number of training patterns is limited and commonly less than the dimension of the feature space. In this paper, a new LDAbased method is proposed. It is based on a straighforward stabilisation approach for the withinclass scatter matrix. In order to evaluate its effectiveness, experiments on face recognition using the wellknown ORL and FERET face databases were carried out and compared with other LDAbased methods. The results indicate that our method improves the LDA classification performance when the withinclass scatter matrix is not only singular but also poorly estimated, with or without a Principal Component Analysis intermediate step and using less linear discriminant features.
Information discriminant analysis: Feature extraction with an informationtheoretic objective
 IEEE Trans. Pattern Anal. Mach. Intell
, 2007
"... Abstract—Using elementary informationtheoretic tools, we develop a novel technique for linear transformation from the space of observations into a lowdimensional (feature) subspace for the purpose of classification. The technique is based on a numerical optimization of an informationtheoretic obj ..."
Abstract

Cited by 23 (7 self)
 Add to MetaCart
Abstract—Using elementary informationtheoretic tools, we develop a novel technique for linear transformation from the space of observations into a lowdimensional (feature) subspace for the purpose of classification. The technique is based on a numerical optimization of an informationtheoretic objective function, which can be computed analytically. The advantages of the proposedmethod over several other techniques are discussed and the conditions underwhich themethod reduces to linear discriminant analysis are given. We show that the novel objective function enjoys many of the properties of the mutual information and the Bayes error and we give sufficient conditions for the method to be Bayesoptimal. Since the objective function is maximized numerically, we show how the calculations can be accelerated to yield feasible solutions. The performance of the method compares favorably to other linear discriminantbased feature extraction methods on a number of simulated and realworld data sets.