Results 1  10
of
43
Efficient and Robust Feature Extraction by Maximum Margin Criterion
 In Advances in Neural Information Processing Systems 16
, 2003
"... In pattern recognition, feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information. Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are two most popular linear dimensionality reduction methods. Howev ..."
Abstract

Cited by 116 (5 self)
 Add to MetaCart
(Show Context)
In pattern recognition, feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information. Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are two most popular linear dimensionality reduction methods. However, PCA is not very effective for the extraction of the most discriminant features and LDA is not stable due to the small sample size problem. In this paper, we propose some new (linear and nonlinear) feature extractors based on maximum margin criterion (MMC). Geometrically, feature extractors based on MMC maximize the (average) margin between classes after dimensionality reduction. It is shown that MMC can represent class separability better than PCA. As a connection to LDA, we may also derive LDA from MMC by incorporating some constraints. By using some other constraints, we establish a new linear feature extractor that does not suffer from the small sample size problem, which is known to cause serious stability problems for LDA. The kernelized (nonlinear) counterpart of this linear feature extractor is also established in the paper. Our extensive experiments demonstrate that the new feature extractors are effective, stable, and efficient.
Computational and Theoretical Analysis of Null Space and Orthogonal Linear Discriminant Analysis
 JOURNAL OF MACHINE LEARNING RESEARCH 7 (2006) 11831204
, 2006
"... Dimensionality reduction is an important preprocessing step in many applications. Linear discriminant analysis (LDA) is a classical statistical approach for supervised dimensionality reduction. It aims to maximize the ratio of the betweenclass distance to the withinclass distance, thus maximizi ..."
Abstract

Cited by 26 (7 self)
 Add to MetaCart
Dimensionality reduction is an important preprocessing step in many applications. Linear discriminant analysis (LDA) is a classical statistical approach for supervised dimensionality reduction. It aims to maximize the ratio of the betweenclass distance to the withinclass distance, thus maximizing the class discrimination. It has been used widely in many applications. However, the classical LDA formulation requires the nonsingularity of the scatter matrices involved. For undersampled problems, where the data dimensionality is much larger than the sample size, all scatter matrices are singular and classical LDA fails. Many extensions, including null space LDA (NLDA) and orthogonal LDA (OLDA), have been proposed in the past to overcome this problem. NLDA aims to maximize the betweenclass distance in the null space of the withinclass scatter matrix, while OLDA computes a set of orthogonal discriminant vectors via the simultaneous diagonalization of the scatter matrices. They have been applied successfully in various applications. In this
Using graph model for face analysis
, 2005
"... Previous work has demonstrated that the image variations of many objects (human faces in particular) under variable lighting can be effectively modelled by low dimensional linear spaces. The typical methods for learning a face subspace include Principal Component Analysis (PCA), Linear Discriminant ..."
Abstract

Cited by 14 (2 self)
 Add to MetaCart
(Show Context)
Previous work has demonstrated that the image variations of many objects (human faces in particular) under variable lighting can be effectively modelled by low dimensional linear spaces. The typical methods for learning a face subspace include Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Locality Preserving Projection (LPP). Theoretical analysis shows that all these three methods can be obtained from different graph models which correspond to different geometrical structures. In this paper, we systematically analyze the relationship between these three subspace methods. We shows that LPP provides a more general framework for subspace learning and a natural solution to the small sample issue in LDA. Extensive experiments on face recognition and clustering are performed on Yale, ORL and PIE databases.
F.: Shaving diffusion tensor images in discriminant analysis: a study into schizophrenia.
 Med. Image Anal.
, 2006
"... Abstract A technique called 'shaving' is introduced to automatically extract the combination of relevant image regions in a comparative study. No hypothesis is needed, as in conventional predefined or expert selected region of interest (ROI)analysis. In contrast to traditional voxel bas ..."
Abstract

Cited by 14 (3 self)
 Add to MetaCart
Abstract A technique called 'shaving' is introduced to automatically extract the combination of relevant image regions in a comparative study. No hypothesis is needed, as in conventional predefined or expert selected region of interest (ROI)analysis. In contrast to traditional voxel based analysis (VBA), correlations within the data can be modeled using principal component analysis (PCA) and linear discriminant analysis (LDA). A study into schizophrenia using diffusion tensor imaging (DTI) serves as an application. Conventional VBA found a decreased fractional anisotropy (FA) in a part of the genu of the corpus callosum and an increased FA in larger parts of white matter. The proposed method reproduced the decrease in FA in the corpus callosum and found an increase in the posterior limb of the internal capsule and uncinate fasciculus. A correlation between the decrease in the corpus callosum and the increase in the uncinate fasciculus was demonstrated.
Null space approach of Fisher discriminant analysis for face recognition
 In Proc. European Conference on Computer Vision, Biometric Authentication Workshop
, 2004
"... Abstract. The null space of the withinclass scatter matrix is found to express most discriminative information for the small sample size problem (SSSP). The null spacebased LDA takes full advantage of the null space while the other methods remove the null space. It proves to be optimal in performa ..."
Abstract

Cited by 12 (2 self)
 Add to MetaCart
(Show Context)
Abstract. The null space of the withinclass scatter matrix is found to express most discriminative information for the small sample size problem (SSSP). The null spacebased LDA takes full advantage of the null space while the other methods remove the null space. It proves to be optimal in performance. From the theoretical analysis, we present the NLDA algorithm and the most suitable situation for NLDA. Our method is simpler than all other null space approaches, it saves the computational cost and maintains the performance simultaneously. Furthermore, kernel technique is incorporated into discriminant analysis in the null space. Firstly, all samples are mapped to the kernel space through a better kernel function, called Cosine kernel, which is proposed to increase the discriminating capability of the original polynomial kernel function. Secondly, a truncated NLDA is employed. The novel approach only requires one eigenvalue analysis and is also applicable to the large sample size problem. Experiments are carried out on different face data sets to demonstrate the effectiveness of the proposed methods. 1
Comments on “Efficient and Robust Feature Extraction by Maximum Margin Criterion”
"... Abstract—The goal of this comment is to first point out two loopholes in the paper by Li et al. (2006): 1) sodesigned efficient maximal margin criterion (MMC) algorithm for small sample size (SSS) problem is problematic and 2) the discussion on the equivalence with the nullspacebased methods in S ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
(Show Context)
Abstract—The goal of this comment is to first point out two loopholes in the paper by Li et al. (2006): 1) sodesigned efficient maximal margin criterion (MMC) algorithm for small sample size (SSS) problem is problematic and 2) the discussion on the equivalence with the nullspacebased methods in SSS problem does not hold. Then, we will present a really efficient MMC algorithm for SSS problem. Index Terms—Efficient algorithm, equivalence, maximal margin criterion (MMC), null space, small sample size (SSS) problem. I. ORGANIZATION AND PREPARATION Organization: In this section, we will give some notations and a brief review of maximum margin criterion (MMC) [3] and point out the two loopholes. In Section II, we will propose a really efficient MMC, and then, conclude this comment in Section III. Let the training set be composed of ™ classes gIYgPY FFFYg™, the �th class have � � training samples, and � � � denote the �th hdimensional sample from the �th class. In total, there will be � a �aI � � training samples. In applications such as face recognition, the small sample size (SSS) problem often takes place, namely, h) �. The withinclass scatter matrix ƒ � and betweenclass scatter matrix ƒ ˜ can be denoted as ƒ � a I ƒ ˜ a I
Nonlinear dimensionality reduction for classification using kernel weighted subspace method
 In Proceedings of the IEEE International Conference on Image Processing
, 2005
"... AbstractWe study the use of kernel subspace methods that learn lowdimensional subspace representations for classification tasks. In particular, we propose a new method called kernel weighted nonlinear discriminant analysis (KWNDA) which possesses several appealing properties. First, like all kern ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
(Show Context)
AbstractWe study the use of kernel subspace methods that learn lowdimensional subspace representations for classification tasks. In particular, we propose a new method called kernel weighted nonlinear discriminant analysis (KWNDA) which possesses several appealing properties. First, like all kernel methods, it handles nonlinearity in a disciplined manner that is also computationally attractive. Second, by introducing weighting functions into the discriminant criterion, it outperforms existing kernel discriminant analysis methods in terms of the classification accuracy. Moreover, it also effectively deals with the small sample size problem. We empirically compare different subspace methods with respect to their classification performance of facial images based on the simple nearest neighbor rule. Experimental results show that KWNDA substantially outperforms competing linear as well as nonlinear subspace methods. 1.
ALLEVIATING THE SMALL SAMPLESIZE PROBLEM IN IVECTOR BASED SPEAKER VERIFICATION
"... This paper investigates the small samplesize problem in ivector based speaker verification systems. The idea of ivectors is to represent the characteristics of speakers in the factors of a factor analyzer. Because the factor loading matrix defines the possible speakerand channelvariability of i ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
This paper investigates the small samplesize problem in ivector based speaker verification systems. The idea of ivectors is to represent the characteristics of speakers in the factors of a factor analyzer. Because the factor loading matrix defines the possible speakerand channelvariability of ivectors, it is important to suppress the unwanted channel variability. Linear discriminant analysis (LDA), withinclass covariance normalization (WCCN), and probabilistic LDA are commonly used for such purpose. These methods, however, require training data comprising many speakers each providing sufficient recording sessions for good performance. Performance will suffer when the number of speakers and/or number of sessions per speaker are too small. This paper compares four approaches to addressing this small samplesize problem: (1) preprocessing the ivectors by PCA before applying LDA (PCA+LDA), (2) replacing the matrix inverse in LDA by pseudoinverse, (3) applying multiway LDA by exploiting the microphone and speaker labels of the training data, and (4) increasing the matrix rank in LDA by generating more ivectors using utterance partitioning. Results based on NIST 2010 SRE suggests that utterance partitioning performs the best, followed by multiway LDA and PCA+LDA. Index Terms — Speaker verification, ivectors, LDA, utterance partitioning, multiway LDA.
Robust and accurate cancer classification with gene expression profiling
 in Proc. 4th IEEE Comput. Syst. Bioinf. Conf
, 2005
"... Robust and accurate cancer classification is critical in cancer treatment. Gene expression profiling is expected to enable us to diagnose tumors precisely and systematically. However, the classification task in this context is very challenging because of the curse of dimensionality and the small sam ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Robust and accurate cancer classification is critical in cancer treatment. Gene expression profiling is expected to enable us to diagnose tumors precisely and systematically. However, the classification task in this context is very challenging because of the curse of dimensionality and the small sample size problem. In this paper, we propose a novel method to solve these two problems. Our method is able to map gene expression data into a very low dimensional space and thus meets the recommended samples to features per class ratio. As a result, it can be used to classify new samples robustly with low and trustable (estimated) error rates. The method is based on linear discriminant analysis (LDA). However, the conventional LDA requires that the withinclass scatter matrix Sw be nonsingular. Unfortunately, Sw is always singular in the case of cancer classification due to the small sample size problem. To overcome this problem, we develop a generalized linear discriminant analysis (GLDA) that is a general, direct, and complete solution to optimize Fisher’s criterion. GLDA is mathematically wellfounded and coincides with the conventional LDA when Sw is nonsingular. Different from the conventional LDA, GLDA does not assume the nonsingularity of Sw, and thus naturally solves the small sample size problem. To accommodate the high dimensionality of scatter matrices, a fast algorithm of GLDA is also developed. Our extensive experiments on seven public cancer datasets show that the method performs well. Especially on some difficult instances that have very small samples to genes per class ratios, our method achieves much higher accuracies than widely used classification methods such as support vector machines, random forests, etc. 1
A study on three Linear Discriminant Analysis based methods in Small Sample Size problem Abstract
"... In this paper, we make a study on three Linear Discriminant Analysis (LDA) based ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
In this paper, we make a study on three Linear Discriminant Analysis (LDA) based