## Rotational Linear Discriminant Analysis Technique for Dimensionality Reduction (under review (2006)

Citations: | 2 - 1 self |

### BibTeX

@MISC{Sharma06rotationallinear,

author = {Alok Sharma and Kuldip K. Paliwal},

title = {Rotational Linear Discriminant Analysis Technique for Dimensionality Reduction (under review},

year = {2006}

}

### OpenURL

### Abstract

Abstract—The linear discriminant analysis (LDA) technique is very popular in pattern recognition for dimensionality reduction. It is a supervised learning technique that finds a linear transformation such that the overlap between the classes is minimum for the projected feature vectors in the reduced feature space. This overlap, if present, adversely affects the classification performance. In this paper, we introduce prior to dimensionality-reduction transformation an additional rotational transform that rotates the feature vectors in the original feature space around their respective class centroids in such a way that the overlap between the classes in the reduced feature space is further minimized. As a result, the classification performance significantly improves, which is demonstrated using several data corpuses. Index Terms—Rotational linear discriminant analysis, dimensionality reduction, classification error, fixed-point algorithm, probability of error. Ç 1

### Citations

3921 |
Pattern Classification and Scene Analysis
- Duda, Hart
- 1973
(Show Context)
Citation Context ...eatures used. In such situations, it becomes important to reduce the dimensionality of the feature space. There are a number of techniques proposed in the literature for dimensionality reduction [3], =-=[7]-=-, [9]; the linear discriminant analysis (LDA) technique [7] is perhaps the most popular among them. LDA is a supervised learning technique that uses a linear transformation to project the feature vect... |

2868 |
P.: UCI Repository of Machine Learning Databases
- Merz, Merphy
- 1996
(Show Context)
Citation Context ...Training Phase of Rotational LDA This section provides an example that illustrates the training phase of the rotational LDA algorithm. For this purpose, the Sat-Image data set from the UCI repository =-=[4]-=- is used. The Sat-Image data set consists of six distinct classes with 36 features. It consists of 4,435 feature vectors for training purposes and 2,000 feature vectors for testing purposes. However, ... |

2649 |
Introduction to Statistical Pattern Recognition”, 2nd edition
- Fukunaga
- 1990
(Show Context)
Citation Context ...es used. In such situations, it becomes important to reduce the dimensionality of the feature space. There are a number of techniques proposed in the literature for dimensionality reduction [3], [7], =-=[9]-=-; the linear discriminant analysis (LDA) technique [7] is perhaps the most popular among them. LDA is a supervised learning technique that uses a linear transformation to project the feature vectors f... |

667 | Statistical pattern recognition: a review
- Jain, Duin, et al.
- 2009
(Show Context)
Citation Context ...age data set [4], Waveform data set [4], TIMIT 2 data set [10], and multiple features (Mfeat) data sets for Karhunen-Loéve coefficients, Fourier coefficients, Zernike moments, and pixel averages [4], =-=[15]-=-. The relevant details of these data sets useful for defining the experimental setup are listed in Table 6. To conduct the experimentation, original feature vectors are reduced from a d-dimensional sp... |

620 | Speaker verification using adapted Gaussian mixture models
- Reynolds, Quatieri, et al.
- 2000
(Show Context)
Citation Context ...erties (or features) from the signal of each frame. In the current state-of-the-art speaker identification systems, the mel frequency cepstral coefficients (MFCCs) are commonly used as features [12], =-=[20]-=-. Thus, a frame is represented by a d-dimensional feature vector with d MFCCs as its components. In the experiment reported in this section, the speaker identification system makes a decision about th... |

512 | Fast and Robust Fixed-Point Algorithms for Independent Component Analysis
- Hyvarinen
- 1999
(Show Context)
Citation Context ...tion. In that case, the rotation matrix can be orthonormalized by using a symmetric orthonormalization procedure starting from a nonorthogonal matrix and continuing the iterative process until T Id d =-=[13]-=-. The value of Perror can be estimated more economically also for each of the iteration of (11) and (12) by applying the following equation instead of applying (1): Perror 1 1 Pc j1 Pc j1 number... |

429 | Oja E: A fast fixed-point algorithm for independent component analysis. Neural Computation
- Hyvärinen
- 1997
(Show Context)
Citation Context ...a should remain unchanged during rotation , the column vectors of should be orthonormal. This means that the square matrix is orthonormal; i.e., T Id d. This allows us to use a fixedpoint algorithm =-=[14]-=-, which is known to be faster and more reliable than the gradient algorithms. The fixed-point algorithm for rotation is written as follows: / Xc j1 n 2 j y j h 1=2 E xðy2R jÞ F x; ; W; xj ; y j i ; ð... |

393 | Using discriminant eigenfeatures for image retrieval
- Swets, Weng
- 1996
(Show Context)
Citation Context ..., [11], [16], which adds a small positive constant to the diagonal elements of SW to make it nonsingular. This method is also suboptimal as Fisher’s criterion is not exactly maximized. Swets and Weng =-=[24]-=- and Belhumeur et al. [2] have proposed a two-stage method (known as the Fisherface method) to avoid the singularity problem. In the first stage, the principal component analysis (PCA) is used for dim... |

303 | Regularized discriminant analysis - Friedman - 1989 |

181 | A Direct LDA Algorithm for High-dimensional Data with Application to Face Recognition
- Yu, Yang, et al.
- 2001
(Show Context)
Citation Context ...formation. In order to avoid this loss in discriminative information, two other recently proposed two-stage methods to overcome the SSS problem are the null space method [6] and the direct LDA method =-=[28]-=-. In the null space method, the first stage transforms the training data vectors to the null space of the within-class scatter matrix SW (i.e., it discards the range space of SW ). In the second stage... |

67 | Discriminative common vectors for face recognition
- Cevikalp, Neamtu, et al.
- 2005
(Show Context)
Citation Context ...Z j1 y2R j pðyj!jÞPð!jÞdy: Fig. 2. Probability error for the two-class problem. Here, we have provided a brief overview of some of the LDA methods to overcome the SSS problem; for other methods, see =-=[5]-=-, [17], [18], [23], and [27]. It is important to note that though these methods overcome the SSS problem, they do not ensure the optimality of Fisher’s criterion as done in the conventional LDA method... |

54 |
Face recognition using LDA-based algorithms
- Lu, Plataniotis, et al.
- 2003
(Show Context)
Citation Context ... pðyj!jÞPð!jÞdy: Fig. 2. Probability error for the two-class problem. Here, we have provided a brief overview of some of the LDA methods to overcome the SSS problem; for other methods, see [5], [17], =-=[18]-=-, [23], and [27]. It is important to note that though these methods overcome the SSS problem, they do not ensure the optimality of Fisher’s criterion as done in the conventional LDA method and hence a... |

49 | Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems
- YE
(Show Context)
Citation Context ... Fig. 2. Probability error for the two-class problem. Here, we have provided a brief overview of some of the LDA methods to overcome the SSS problem; for other methods, see [5], [17], [18], [23], and =-=[27]-=-. It is important to note that though these methods overcome the SSS problem, they do not ensure the optimality of Fisher’s criterion as done in the conventional LDA method and hence are suboptimal. 3... |

44 |
Discriminant analysis with singular covariance matrices: Methods and applications to spectroscopic data
- Krzanowski, Jonathan, et al.
- 1995
(Show Context)
Citation Context ...y its pseudoinverse [19], [25]. However, this does not guarantee the optimality of Fisher’s criterion. Another way to overcome the singularity problem is through the regularized LDA method [8], [11], =-=[16]-=-, which adds a small positive constant to the diagonal elements of SW to make it nonsingular. This method is also suboptimal as Fisher’s criterion is not exactly maximized. Swets and Weng [24] and Bel... |

42 |
Expected classification error of the fisher linear classifier with pseudo-inverse covariance matrix
- Raudys, Duin
- 1998
(Show Context)
Citation Context ...thods have been proposed in the literature to overcome this problem. A simple and direct way to avoid the singularity problem is to replace the inverse of the within-class matrix by its pseudoinverse =-=[19]-=-, [25]. However, this does not guarantee the optimality of Fisher’s criterion. Another way to overcome the singularity problem is through the regularized LDA method [8], [11], [16], which adds a small... |

41 |
Eigenfaces versus Fisherfaces: recognition sing class specific linear projection
- Belhumeur, Hespanha, et al.
- 1997
(Show Context)
Citation Context ... small positive constant to the diagonal elements of SW to make it nonsingular. This method is also suboptimal as Fisher’s criterion is not exactly maximized. Swets and Weng [24] and Belhumeur et al. =-=[2]-=- have proposed a two-stage method (known as the Fisherface method) to avoid the singularity problem. In the first stage, the principal component analysis (PCA) is used for dimensionality reduction in ... |

34 |
Fractional-step dimensionality reduction
- Lotlikar, Kothari
- 2000
(Show Context)
Citation Context ... y2R j pðyj!jÞPð!jÞdy: Fig. 2. Probability error for the two-class problem. Here, we have provided a brief overview of some of the LDA methods to overcome the SSS problem; for other methods, see [5], =-=[17]-=-, [18], [23], and [27]. It is important to note that though these methods overcome the SSS problem, they do not ensure the optimality of Fisher’s criterion as done in the conventional LDA method and h... |

21 |
Image classification by the Foley-Sammon transform
- Tian, Barbero, et al.
- 1986
(Show Context)
Citation Context ...have been proposed in the literature to overcome this problem. A simple and direct way to avoid the singularity problem is to replace the inverse of the within-class matrix by its pseudoinverse [19], =-=[25]-=-. However, this does not guarantee the optimality of Fisher’s criterion. Another way to overcome the singularity problem is through the regularized LDA method [8], [11], [16], which adds a small posit... |

8 |
A New LDA-based Face Recognition
- Chen, Liao, et al.
- 2000
(Show Context)
Citation Context ...e loses some discriminative information. In order to avoid this loss in discriminative information, two other recently proposed two-stage methods to overcome the SSS problem are the null space method =-=[6]-=- and the direct LDA method [28]. In the null space method, the first stage transforms the training data vectors to the null space of the within-class scatter matrix SW (i.e., it discards the range spa... |

8 |
Identity Verification Using Speech and Face
- Sanderson, Paliwal
- 2004
(Show Context)
Citation Context ...rom a given utterance belong to the same class. We use the mean of the L neighboring frames as an estimate of the class centroid. Note that a similar situation occurs in face recognition applications =-=[21]-=-, where one can find the class-centroid information by dividing a given image of a face into blocks, representing each block by its local features and using the neighboring blocks to estimate the clas... |

4 | A gradient linear discriminant analysis for small sample sized problem
- Sharma, Paliwal
- 2008
(Show Context)
Citation Context ...jÞPð!jÞdy: Fig. 2. Probability error for the two-class problem. Here, we have provided a brief overview of some of the LDA methods to overcome the SSS problem; for other methods, see [5], [17], [18], =-=[23]-=-, and [27]. It is important to note that though these methods overcome the SSS problem, they do not ensure the optimality of Fisher’s criterion as done in the conventional LDA method and hence are sub... |

2 |
G.C.: Class-dependent PCA, LDA and MDC: a combined classifier for pattern classification
- Sharma, Paliwal, et al.
- 2006
(Show Context)
Citation Context ...ime in computing these matrices for each of the iterations. It should be noted here that though we have used orientation W from the LDA technique, one could apply some other techniques instead of LDA =-=[22]-=- together with the rotational method for the improvement of recognition and/or classification. In that case, the optimization criteria will be different, depending upon the technique then used. The se... |

1 |
Regularized Discriminant Analysis and Its Application in
- Guo, Hastie, et al.
- 2007
(Show Context)
Citation Context ...trix by its pseudoinverse [19], [25]. However, this does not guarantee the optimality of Fisher’s criterion. Another way to overcome the singularity problem is through the regularized LDA method [8], =-=[11]-=-, [16], which adds a small positive constant to the diagonal elements of SW to make it nonsingular. This method is also suboptimal as Fisher’s criterion is not exactly maximized. Swets and Weng [24] a... |

1 |
The HTK Book Version 3.2. Cambridge Univ
- Young, Evermann, et al.
- 2002
(Show Context)
Citation Context ...els are extracted; then, each vowel is divided into three segments, and each segment is used in getting mel-frequency cepstral coefficients with energy-delta-acceleration (MFCC_E_D_A) feature vectors =-=[26]-=-. The minimum classification error for the Sat-Image data set produced by LDA is 19.2 percent, whereas it is only 1.1 percent by the rotational LDA algorithm. Similarly, for Mfeat-Fourier coefficients... |