Efficient Clustering Of Very Large Document Collections

Efficient Clustering Of Very Large Document Collections (2001)

by Inderjit S. Dhillon , James Fan , Yuqiang Guan

Citations:

74 - 9 self

User correction supplied by SystemCorrections

Datum	Value	Source
TITLE	Efficient Clustering Of Very Large Document Collections	user correction - Legacy Corrections
AUTHOR NAME	Inderjit S. Dhillon	SVM HeaderParse 0.1
AUTHOR NAME	James Fan	SVM HeaderParse 0.1
AUTHOR NAME	Yuqiang Guan	SVM HeaderParse 0.1
ABSTRACT	An invaluable portion of scientific data occurs naturally in text form. Given a large unlabeled document collection, it is often helpful to organize this collection into clusters of related documents. By using a vector space model, text data can be treated as high-dimensional but sparse numerical data vectors. It is a contemporary challenge to efficiently preprocess and cluster very large document collections. In this paper we present a time and memory ecient technique for the entire clustering process, including the creation of the vector space model. This efficiency is obtained by (i) a memory-ecient multi-threaded preprocessing scheme, and (ii) a fast clustering algorithm that fully exploits the sparsity of the data set. We show that this entire process takes time that is linear in the size of the document collection. Detailed experimental results are presented - a highlight of our results is that we are able to effectively cluster a collection of 113,716 NSF award abstracts in 23 minutes (including disk I/O costs) on a single workstation with modest memory consumption.	user correction - Legacy Corrections
YEAR	2001	user correction - Legacy Corrections
CITATIONS	23 found	ParsCit 1.0