Abstract

High dimensional data has always been a challenge for clustering algorithms because of the inherent sparsity of the points. Recent research results indicate that in high dimensional data, even the concept of proximity or clustering may not be meaningful. We discuss very general techniques for projected clustering which are able to construct clusters in arbitrarily aligned subspaces of lower dimensionality. The subspaces are specific to the clusters themselves. This definition is substantially more general and realistic than currently available techniques which limit the method to only projections from the original set of attributes. The generalized projected clustering technique may also be viewed as a way of trying to rede ne clustering for high dimensional applications by searching for hidden subspaces with clusters which are created by inter-attribute correlations. We provide a new concept of using extended cluster feature vectors in order to make the algorithm scalable for very large databases. The running time and space requirements of the algorithm are adjustable, and are likely to tradeoff with better accuracy.

Citations

1753	Dubes, Algorithms for Clustering Data - Jain, C - 1988
1437	I: Principal Component Analysis - Jolliffe - 1986
533	Approximate nearest neighbors: towards removing the curse of dimensionality - Indyk, Motwani - 1998
461	Automatic subspace clustering of high dimensional data - Agrawal, Gehrke, et al. - 2005
432	CURE: an efficient clustering algorithm for large databases - Guha, Rastogi, et al. - 1998
368	ªFastmap: A Fast Algorithm for Indexing, Data-Mining and - Faloutsos, Lin - 1995
222	When is “nearest neighbor” meaningful - Beyer, Goldstein, et al. - 1999
196	Fast algorithms for projected clustering - Aggarwal, Wolf, et al. - 1999
192	An efficient data clustering method for very large databases - Birch - 1996
150	Two algorithms for nearest-neighbor search in high dimensions - Kleinberg - 1997
89	Entropy-based subspace clustering for mining numerical data - Cheng, Fu, et al. - 1999
88	Dimensionality reduction for similarity searching in dynamic databases - Kanth, Agrawal, et al. - 1998
74	Optimal grid-clustering: Towards breaking the curse of dimensionality in high-dimensional clustering - Hinneburg, Keim - 1999
55	A distribution-based clustering algorithm for mining in large spatial databases - XU, ESTER, et al. - 1998
46	D.: “Feature Subset Selection Using the Wrapper Method: Overtting and Dynamic Search Space Topology - Kohavi, Sommereld - 1995
35	A Density- based Algorithm for Discovering Clusters in Large Spatial Databases with Noise - Ester, Kriegel, et al. - 1996
21	Jolliffe Principal Component Analysis - T - 1986
12	CURE: An E cient Clustering Algorithm for Large Databases - Guha, Rastogi, et al. - 1998
3	Automatic subspace clustering of high dimensional data for data mining applications - Al - 1998
2	Fast Algorithms for Projected Clustering - al - 1999
2	Efficient and Effective Clustering Methods for - Ng, Han - 1994
1	When is nearest neighbor meaningful - al - 1999
1	A Density Based Algorithm for Discovering Clusters - al - 1996
1	A Distribution-Based Clustering Algorithm for Mining - al - 1998
1	E cient and E ective Clustering Methods for - Ng, Han - 1994