and Laplacianfaces (LAP) are three recently proposed methods which can effectively learn linear projection matrices for dimensionality reduction in face recognition, where the dimension of the sample space is typically larger than the number of samples in the training set and consequently the so-called small sample size (SSS) problem exists. The three methods obtained their respective projection matrices based on different objective functions and all claimed to be superior to such methods as Principal Component Analysis (PCA) and PCA plus Linear Discriminant Analysis (PCA+LDA) in terms of classification accuracy. However, in literature, no comparative study is carried out among them. In this paper, we carry out a comparative study among them in face recognition (or generally in the SSS problem), and argue that the projection matrix yielded by DCV is the optimal solution to both NCA and LAP in terms of their respective objective functions, whereas neither NCA nor LAP may get their own optimal solutions. In addition, we show that DCV is more efficient than both NCA and LAP for both linear dimensionality reduction and subsequent classification in SSS problem. Finally, experiments are conducted on ORL, AR and YALE face databases to verify our arguments and to present some insights for future study.

Abstract not found

Abstract. Principal Component Analysis (PCA) plus Linear Discriminant Analysis (LDA) (PCA+LDA) and LDA/QR are both two-stage methods that deal with the small sample size (SSS) problem in traditional LDA. When applied to face recognition under varying lighting conditions and different facial expressions, neither method may work robustly due to limited number of training samples for each class in the training set. Recently, resampling, a technique that generates multiple subsets of samples from the training set, has been successfully employed to improve the classification performance of the PCA+LDA classifier. In this paper, stimulated by such success, we propose a resampling LDA/QR method to improve LDA/QR’s performance. Furthermore, by analyzing the difference between LDA/QR and PCA+LDA and taking advantage of such difference, we incorporate LDA/QR and PCA+LDA in a combined framework by resampling for face recognition. Experimental results on AR dataset show that 1) resampling LDA/QR yields significantly higher classification performance than the original LDA/QR, and 2) resampling LDA/QR and resampling PCA+LDA in a combined framework further improves the classification compared to either resampling LDA/QR or resampling PCA+LDA. 1

Abstract. Pseudoinverse Linear Discriminant Analysis (PLDA) is a classical and pioneer method that deals with the Small Sample Size (SSS) problem in LDA when applied to such application as face recognition. However, it is expensive in computation and storage due to manipulating on extremely large d × d matrices, where d is the dimensionality of the sample image. As a result, although frequently cited in literature, PLDA is hardly compared in terms of classification performance with the newly proposed methods. In this paper, we propose a new feature extraction method named RSw+LDA, which is 1) much more efficient than PLDA in both computation and storage; and 2) theoretically equivalent to PLDA, meaning that it produces the same projection matrix as PLDA. Our experimental results on AR face dataset, a challenging dataset with variations in expression, lighting and occlusion, show that PLDA (or RSw+LDA) can achieve significantly higher classification accuracy than the recently proposed Linear Discriminant Analysis via QR decomposition and Discriminant Common Vectors.

In this paper, we develop a geometric framework for linear or nonlinear discriminant subspace learning and classification. In our framework, the structures of classes are conceptualized as a semi-Riemannian manifold which is considered as a submanifold embedded in an ambient semi-Riemannian space. The class structures of original samples can be characterized and deformed by local metrics of the semi-Riemannian space. Semi-Riemannian metrics are uniquely determined by the smoothing of discrete functions and the nullity of the semi-Riemannian space. Based on the geometrization of class structures, optimizing class structures in the feature space is equivalent to maximizing the quadratic quantities of metric tensors in the semi-Riemannian space. Thus supervised discriminant subspace learning reduces to unsupervised semi-Riemannian manifold learning. Based on the proposed framework, a novel algorithm, dubbed as Semi-Riemannian Discriminant Analysis (SRDA), is presented for subspace-based classification. The performance of SRDA is tested on face recognition (singular case) and handwritten capital letter classification (nonsingular case) against existing algorithms. The experimental results show that SRDA works well on recognition and classification, implying that semi-Riemannian geometry is a promising new tool for pattern recognition and machine learning. 1.

Face Representation (FR) plays a typically important role in face recognition and methods such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) have been received wide attention recently. However, despite of the achieved successes, these FR methods will inevitably lead to poor classification performance in case of great facial variations such as expression, lighting, occlusion and so on, due to the fact that the image gray value matrices on which they manipulate are very sensitive to these facial variations. In this paper, we take notice of the facts that every image matrix can always have the well-known Singular Value Decomposition (SVD) and can be regarded as a composition of a set of base images generated by SVD, and we further point out that the leading base images (those corresponding to large singular values) on one hand are sensitive to the aforementioned facial variations and on the other hand dominate the composition of the face image. Then based on these observations, we subtly deflate the weights of the facial variation sensitive base images by a parameter α and propose a novel Fractional order Singular Value Decomposition Representation (FSVDR) to alleviate facial variations for face recognition. Finally, our experimental results show that FSVDR can: 1) effectively alleviate facial variations; and 2) form an intermediate representation for many FR methods such as PCA and LDA to significantly improve their classification performance in case of great facial variations.

Linear techniques are widely used to reduce the dimension of image representa-tion spaces in applications such as image indexing and object recognition. Optimal Component Analysis (OCA) is a method that addresses the problem of learning an optimal linear representation for a particular classification task. The problem is formulated in the framework of optimization on a Grassmann manifold and treated with stochastic gradient methods intrinsic to the manifold. OCA has been suc-cessfully applied to image classification problems arising in a variety of contexts. However, as the search space is typically very high dimensional, OCA optimization often requires a large number of iterations, each involving extensive computations that make the algorithm somewhat costly to implement. In this paper, we propose a two-stage method, which we refer to as two-stage OCA, that improves the search efficiency by orders of magnitude without compromising the quality of the esti-mation. In fact, extensive experiments using face and object classification datasets indicate that the proposed method often leads to more accurate classification than the original OCA since it is not as prone to over-fitting. Two-stage OCA also leads to Preprint submitted to Elsevier 30 May 2007 substantial improvement in classification performance as compared to other linear dimension reduction methods.

Feature extraction techniques are widely used to reduce the complexity high dimensional data. Nonlinear feature extraction via Locally Linear Embedding (LLE) has attracted much attention due to their high performance. In this paper, we proposed a novel approach for face recognition to address the challenging task of recognition using integration of nonlinear dimensional reduction Locally Linear Embedding integrated with Local Fisher Discriminant Analysis (LFDA) to improve the discriminating power of the extracted features by maximize between-class while within-class local structure is preserved. Extensive experimentation performed on the CMU-PIE database indicates that the proposed methodology outperforms Benchmark methods such as Principal Component Analysis (PCA), Fisher Discrimination Analysis (FDA). The results showed that 95 % of recognition rate could be obtained using our proposed method.

Spike sorting is of prime importance in neurophysiology and hence has received considerable attention. However, conventional methods suffer from the degradation of clustering results in the presence of high levels of noise contamination. This paper presents a scheme for taking advantage of automatic clustering and enhancing the feature extraction efficiency, especially for low-SNR spike data. The method employs linear discriminant analysis based on a fuzzy c-means (FCM) algorithm. Simulated spike data [1] were used as the test bed due to better a priori knowledge of the spike signals. Application to both high and low signal-to-noise ratio (SNR) data showed that the proposed method outperforms conventional principal-component analysis (PCA) and FCM algorithm. FCM failed to cluster spikes for low-SNR data. For two discriminative performance indices based on Fisher's discriminant criterion, the proposed approach was over 1.36 times the ratio of between- and within-class variation of PCA for spike data with SNR ranging from 1.5 to 4.5 dB. In conclusion, the proposed scheme is unsupervised and can enhance the performance of fuzzy c-means clustering in spike sorting with low-SNR data. Keywords: Spike sorting; spike classification; fuzzy c-means; principal-component analysis; linear discriminant analysis; low-SNR.

Abstract Tensor representation is helpful to reduce the small sample size problem in discriminative subspace selection. As pointed by this paper, this is mainly because the structure information of objects in computer vision research is a reasonable constraint to reduce the number of unknown parameters used to represent a learning model. Therefore, we apply this information to the vector-based learning and generalize the vector-based learning to the tensor-based learning as the supervised tensor learning (STL) framework, which accepts tensors as input. To obtain the solution of STL, the alternating projection optimization procedure is developed. The STL framework is a combination of the convex optimization and the operations in multilinear algebra. The tensor representation helps reduce the overfitting problem in vector-based learning. Based on STL and its alternating projection optimization procedure, we generalize support vector machines, minimax probability machine, Fisher discriminant analysis, and distance metric learning, to support tensor machines, tensor minimax probability machine, tensor Fisher discriminant analysis, and the multiple distance metrics learning, respectively. We also study the iterative procedure for feature extraction within STL. To examine the effectiveness of STL, we implement the tensor minimax probability machine for image classification. By comparing with minimax probability machine, the tensor version reduces the overfitting problem.