SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis

DMCA

SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis (2008)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Deng Cai , Student Member , Xiaofei He , Jiawei Han , Senior Member

Venue:	IEEE Transactions on Knowledge and Data Engineering
Citations:	32 - 1 self

BibTeX

@ARTICLE{Cai08srda:an,
    author = {Deng Cai and Student Member and Xiaofei He and Jiawei Han and Senior Member},
    title = {SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis},
    journal = {IEEE Transactions on Knowledge and Data Engineering},
    year = {2008},
    pages = {1--12}
}

OpenURL

Abstract

Abstract—Linear Discriminant Analysis (LDA) has been a popular method for extracting features that preserves class separability. The projection functions of LDA are commonly obtained by maximizing the between-class covariance and simultaneously minimizing the within-class covariance. It has been widely used in many fields of information processing, such as machine learning, data mining, information retrieval, and pattern recognition. However, the computation of LDA involves dense matrices eigendecomposition, which can be computationally expensive in both time and memory. Specifically, LDA has Oðmnt þ t3Þ time complexity and requires Oðmn þ mt þ ntÞ memory, where m is the number of samples, n is the number of features, and t minðm; nÞ. When both m and n are large, it is infeasible to apply LDA. In this paper, we propose a novel algorithm for discriminant analysis, called Spectral Regression Discriminant Analysis (SRDA). By using spectral graph analysis, SRDA casts discriminant analysis into a regression framework that facilitates both efficient computation and the use of regularization techniques. Specifically, SRDA only needs to solve a set of regularized least squares problems, and there is no eigenvector computation involved, which is a huge save of both time and memory. Our theoretical analysis shows that SRDA can be computed with OðmsÞ time and OðmsÞ memory, where sð nÞ is the average number of nonzero features in each sample. Extensive experimental results on four real-world data sets demonstrate the effectiveness and efficiency of our algorithm. Index Terms—Linear Discriminant Analysis, spectral regression, dimensionality reduction. Ç 1

Keyphrases

large-scale discriminant analysis efficient algorithm discriminant analysis data mining information retrieval spectral graph analysis many field theoretical analysis regression framework mn mt nt memory regularization technique between-class covariance abstract linear discriminant analysis dense matrix eigendecomposition class separability spectral regression square problem average number dimensionality reduction huge save within-class covariance index term real-world data set spectral regression discriminant analysis nonzero feature eigenvector computation popular method m time novel algorithm extensive experimental result pattern recognition efficient computation information processing projection function machine learning m memory mnt t3 time complexity

DMCA