We derive a class of computationally inexpensive linear dimension reduction criteria by introducing a weighted variant of the well-known K-class Fisher criterion associated with linear discriminant analysis (LDA). It can be seen that LDA weights contributions of individual class pairs according to the Euclidian distance of the respective class means. We generalize upon LDA by introducing a different weighting function.

vii CHAPTER 1:

In this paper, we present counterarguments against the direct LDA algorithm (D-LDA), which was previously claimed to be equivalent to Linear Discriminant Analysis (LDA). We show from Bayesian decision theory that D-LDA is actually a special case of LDA by directly taking the linear space of class means as the LDA solution. The pooled covariance estimate is completely ignored. Furthermore, we demonstrate that D-LDA is not equivalent to traditional subspace-based LDA in dealing with the Small Sample Size problem. As a result, D-LDA may impose a significant performance limitation in general applications.

This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by sending a blank email message to

Linear Disciminant Analysis is in general unable to find the lower-dimensional feature space which maximizes the class discrimination, even if the class distributions can be assumed to be very simple, e.g. Gaussians with identical covariance matrices. In this paper we reformulate the #-class Fisher criterion as a sum of ### #####-class Fisher criteria. This formulation allows to weigh class pair contributions according to their relevance for classification. Further it offers an obvious way how to cope with heteroscedastic models. We propose a particular weighting scheme which attempts to approximate the pairwise Bayes error. Moderate improvements are obtained on the TIMIT phoneme classification task.