## Combining multiple clusterings using evidence accumulation (2005)

Venue: | IEEE Transaction on Pattern Analysis and Machine Intelligence |

Citations: | 60 - 5 self |

### BibTeX

@ARTICLE{Fred05combiningmultiple,

author = {Ana L. N. Fred and Anil K. Jain},

title = {Combining multiple clusterings using evidence accumulation},

journal = {IEEE Transaction on Pattern Analysis and Machine Intelligence},

year = {2005},

volume = {27},

pages = {835--850}

}

### OpenURL

### Abstract

We explore the idea of evidence accumulation (EAC) for combining the results of multiple clusterings. First, a clustering ensemble- a set of object partitions, is produced. Given a data set (n objects or patterns in d dimensions), different ways of producing data partitions are: (1)- applying different clustering algorithms, and (2)- applying the same clustering algorithm with different values of parameters or initializations. Further, combinations of different data representations (feature spaces) and clustering algorithms can also provide a multitude of significantly different data partitionings. We propose a simple framework for extracting a consistent clustering, given the various partitions in a clustering ensemble. According to the EAC concept, each partition is viewed as an independent evidence of data organization, individual data partitions being combined, based on a voting mechanism, to generate a new n × n similarity matrix between the n patterns. The final data partition of the n patterns is obtained by applying a hierarchical agglomerative clustering algorithm on this matrix. We have developed a theoretical framework for the analysis of the proposed clustering combination strategy and its evaluation, based on the concept of mutual information between data partitions. Stability of the results is evaluated using bootstrapping techniques. A detailed discussion of an evidence accumulation-based clustering algorithm, using a split and merge strategy based on the K-means clustering algorithm, is presented. Experimental results of the proposed method on several synthetic and real data sets are compared with other combination strategies, and with individual clustering results produced by well known clustering algorithms.

### Citations

9231 |
Elements of Information Theory
- Cover, Thomas
- 1990
(Show Context)
Citation Context ...rtitions Using Mutual Information A partition P a describes a labelling of the n patterns in the data set X, into ka clusters. Taking frequency counts as approximations for probabilities, the entropy =-=[58]-=- of the data partition P a is expressed by H(P a ) = − �ka n i=1 a i n log � n a i n � , where n a i represents the number of patterns in cluster C a i ∈ P a . The agreement between two partitions P a... |

2928 |
Introduction to Statistical Pattern Recognition, Electrical Science Series
- Fukunaga
- 1972
(Show Context)
Citation Context ... a pairwise clustering cost function, emphasizing cluster connectedness. Non-parametric density based clustering methods attempt to identify high density clusters separated by low density regions [5] =-=[32]-=-, [33]. Graph-theoretical approaches [34] have mostly been explored in hierarchical methods, that can be represented graphically as a tree or dendrogram [7], [8]. Both agglomerative [28], [35] and div... |

2775 | Normalized cuts and image segmentation
- SHI, MALIK
- 1997
(Show Context)
Citation Context ...idity [39], [5], [40], [41], [42], [43]. Treating the clustering problem as a graph partitioning problem, a recent approach, known as spectral clustering, applies spectral graph theory for clustering =-=[44]-=-, [45], [46]. Among the various clustering methods, the K-means algorithm, which minimizes the squared-error criteria, is one of the simplest clustering algorithm. It is computationally efficient and ... |

2334 |
Algorithms for Clustering Data
- Jain, Dubes
- 1988
(Show Context)
Citation Context ...nclude: parametric density approaches, such as mixture decomposition techniques [23], [24], [25], [26]; prototype-based methods, such as central clustering [14], square-error clustering [27], K-means =-=[28]-=-, [8] or K-medoids clustering [9]; and shape fitting approaches [15], [6], [16]. Model order selection is sometimes left as a design parameter to be specified by the user, or it is incorporated in the... |

1535 |
Finding Groups in Data: An Introduction to Cluster Analysis
- Kaufman, Rousseeuw
- 1990
(Show Context)
Citation Context ...ter shapes. Additionally, quantitative evaluation of the quality of clustering results is difficult due to the subjective notion of clustering. A large number of clustering algorithms exist [7], [8], =-=[9]-=-, [10], [11], yet no single algorithm is able to identify all sorts of cluster shapes and structures that are encountered in practice. Each algorithm has its own approach for estimating the number of ... |

1413 | Data clustering: a review
- Jain, Murty, et al.
- 1999
(Show Context)
Citation Context ...about cluster shapes. Additionally, quantitative evaluation of the quality of clustering results is difficult due to the subjective notion of clustering. A large number of clustering algorithms exist =-=[7]-=-, [8], [9], [10], [11], yet no single algorithm is able to identify all sorts of cluster shapes and structures that are encountered in practice. Each algorithm has its own approach for estimating the ... |

1186 | On spectral clustering: Analysis and an algorithm
- Ng, Jordan, et al.
- 2002
(Show Context)
Citation Context ...[39], [5], [40], [41], [42], [43]. Treating the clustering problem as a graph partitioning problem, a recent approach, known as spectral clustering, applies spectral graph theory for clustering [44], =-=[45]-=-, [46]. Among the various clustering methods, the K-means algorithm, which minimizes the squared-error criteria, is one of the simplest clustering algorithm. It is computationally efficient and does n... |

1102 | On combining classifiers
- Kittler, Hatef, et al.
- 1998
(Show Context)
Citation Context ...sis, it is wise to apply several different clustering algorithms to the given data and then determine the best algorithm for the data. Inspired by the work in sensor fusion and classifier combination =-=[50]-=-, [51], [52], a clustering combination approach has been proposed [53], [54], [55]. Fred and Jain introduce the concept of evidence accumulation clustering, that maps the individual data partitions in... |

588 | CURE: an efficient clustering algorithm for large databases
- Guha, Rastogi, et al.
- 1998
(Show Context)
Citation Context ...er representatives, such as the centroid, emphasize compactness. Variations of thesprototype-based hierarchical clustering include the use of multiple prototypes per cluster, as in the CURE algorithm =-=[38]-=-. Other hierarchical agglomerative clustering algorithms follow a split and merge technique, the data being initially split into a large number of small clusters, merging being based on inter-cluster ... |

548 |
Cluster Analysis
- Everitt, Landau, et al.
- 2001
(Show Context)
Citation Context ...hapes. Additionally, quantitative evaluation of the quality of clustering results is difficult due to the subjective notion of clustering. A large number of clustering algorithms exist [7], [8], [9], =-=[10]-=-, [11], yet no single algorithm is able to identify all sorts of cluster shapes and structures that are encountered in practice. Each algorithm has its own approach for estimating the number of cluste... |

516 |
Mixture Models: Inference and Applications to Clustering
- McLachlan, Basford
- 1988
(Show Context)
Citation Context ...ssume particular cluster shapes that can be given a simple and compact description. Examples of model-based techniques include: parametric density approaches, such as mixture decomposition techniques =-=[23]-=-, [24], [25], [26]; prototype-based methods, such as central clustering [14], square-error clustering [27], K-means [28], [8] or K-medoids clustering [9]; and shape fitting approaches [15], [6], [16].... |

503 |
Pattern Recognition
- Theodoridis, Koutroumbas
- 1999
(Show Context)
Citation Context ... Additionally, quantitative evaluation of the quality of clustering results is difficult due to the subjective notion of clustering. A large number of clustering algorithms exist [7], [8], [9], [10], =-=[11]-=-, yet no single algorithm is able to identify all sorts of cluster shapes and structures that are encountered in practice. Each algorithm has its own approach for estimating the number of clusters [12... |

429 | Cluster ensembles: A knowledge Reuse Framework for Combining Partitionings
- Strehl, Ghosh
- 2004
(Show Context)
Citation Context ...ng algorithms such as the K-means, which impose a simple structure on the data, can lead to the identification of true underlying clusters with arbitrary shapes, sizes and densities. Strehl and Ghosh =-=[56]-=- explore the concept of consensus between data partitions and propose three different combination mechanisms. The first step of the consensus functions is to transform the data partitions into a hyper... |

428 | shift, mode seeking, and clustering
- Cheng, “Mean
- 1995
(Show Context)
Citation Context ... clusters, merging being based on inter-cluster similarity; a final partition is selected among the clustering hierarchy by thresholding techniques or based on measures of cluster validity [39], [5], =-=[40]-=-, [41], [42], [43]. Treating the clustering problem as a graph partitioning problem, a recent approach, known as spectral clustering, applies spectral graph theory for clustering [44], [45], [46]. Amo... |

350 |
Model-based Gaussian and non-Gaussian clustering
- Banfield, Raftery
- 1993
(Show Context)
Citation Context ...luster shapes that can be given a simple and compact description. Examples of model-based techniques include: parametric density approaches, such as mixture decomposition techniques [23], [24], [25], =-=[26]-=-; prototype-based methods, such as central clustering [14], square-error clustering [27], K-means [28], [8] or K-medoids clustering [9]; and shape fitting approaches [15], [6], [16]. Model order selec... |

316 | How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis.” The Computer Journal 41(8):578–88
- Fraley, Raftery
- 1998
(Show Context)
Citation Context ...thms exist, it is difficult to find a single clustering algorithm that can handle all types of cluster shapes and sizes, or even decide which algorithm would be the best one for a particular data set =-=[48]-=-, [49]. Figure 1 illustrates how different algorithms, or even the same algorithm with different parameters, produce very distinct results. Considering that clustering is an important tool for data mi... |

304 | Unsupervised learning on finite mixture models
- Figueiredo, Jain
- 2002
(Show Context)
Citation Context ...ular cluster shapes that can be given a simple and compact description. Examples of model-based techniques include: parametric density approaches, such as mixture decomposition techniques [23], [24], =-=[25]-=-, [26]; prototype-based methods, such as central clustering [14], square-error clustering [27], K-means [28], [8] or K-medoids clustering [9]; and shape fitting approaches [15], [6], [16]. Model order... |

264 | Graph-theoretical methods for detecting and describing gestalt clusters
- Zahn
- 1971
(Show Context)
Citation Context ...hasizing cluster connectedness. Non-parametric density based clustering methods attempt to identify high density clusters separated by low density regions [5] [32], [33]. Graph-theoretical approaches =-=[34]-=- have mostly been explored in hierarchical methods, that can be represented graphically as a tree or dendrogram [7], [8]. Both agglomerative [28], [35] and divisive approaches [36] (such as those base... |

223 | a hierarchical clustering algorithm using dynamic modeling
- Karypis, Han, et al.
- 1999
(Show Context)
Citation Context ...erging being based on inter-cluster similarity; a final partition is selected among the clustering hierarchy by thresholding techniques or based on measures of cluster validity [39], [5], [40], [41], =-=[42]-=-, [43]. Treating the clustering problem as a graph partitioning problem, a recent approach, known as spectral clustering, applies spectral graph theory for clustering [44], [45], [46]. Among the vario... |

131 | Clustering using a similarity measure based on Shared Nearest Neighbors
- Jarvis, Patrick
- 1973
(Show Context)
Citation Context ...natural clusters to be less clearly defined (see fig. 9(c) for k = 80); as a result,sthe number of clusters obtained in the combined partition increases with k. (a)k ∈ [2; 10]. (b)k ∈ [2; 20]. (c)k ∈ =-=[60; 90]-=-. (d)k ∈ [2; 80]. Fig. 10. Combining 200 K-means clusterings, with k randomly selected within an interval [kmin, kmax]. Each figure shows the dendrogram produced by the SL method over the co-associati... |

103 | Data Clustering Using Evidence Accumulation
- Fred, Jain
- 2002
(Show Context)
Citation Context ...n data and then determine the best algorithm for the data. Inspired by the work in sensor fusion and classifier combination [50], [51], [52], a clustering combination approach has been proposed [53], =-=[54]-=-, [55]. Fred and Jain introduce the concept of evidence accumulation clustering, that maps the individual data partitions in a clustering ensemble into a new similarity measure between patterns, summa... |

94 | ªA Robust Competitive Clustering Algorithm with Applications in Computer Vision,º
- Frigui, Krishnapuram
- 1999
(Show Context)
Citation Context ...lusters. A number of application areas use clustering techniques for organizing or discovering structure in data, such as data mining [1], [2], information retrieval [3], [4], [5], image segmentation =-=[6]-=-, and machine learning. In real world problems, clusters can appear with different shapes, sizes, data sparseness, and degree of separation. Further, noise in the data can mask the true underlying str... |

93 |
On cluster validity for the fuzzy c-means model
- Pal, Bezdek
- 1995
(Show Context)
Citation Context ...ctice. Each algorithm has its own approach for estimating the number of clusters [12], [13], imposing a structure on the data [14], [15], [16], and validating the resulting clusters [17], [18], [19], =-=[20]-=-, [21], [22]. Model-based techniques assume particular cluster shapes that can be given a simple and compact description. Examples of model-based techniques include: parametric density approaches, suc... |

81 | Bayesian approaches to gaussian mixture modelling
- Roberts, Husmeier, et al.
- 1998
(Show Context)
Citation Context ...particular cluster shapes that can be given a simple and compact description. Examples of model-based techniques include: parametric density approaches, such as mixture decomposition techniques [23], =-=[24]-=-, [25], [26]; prototype-based methods, such as central clustering [14], square-error clustering [27], K-means [28], [8] or K-medoids clustering [9]; and shape fitting approaches [15], [6], [16]. Model... |

79 | ªA Lattice Conceptual Clustering System and Its Application to Browsing Retrieval,º
- Carpineto, Romano
- 1996
(Show Context)
Citation Context ... into homogeneous groups or clusters. A number of application areas use clustering techniques for organizing or discovering structure in data, such as data mining [1], [2], information retrieval [3], =-=[4]-=-, [5], image segmentation [6], and machine learning. In real world problems, clusters can appear with different shapes, sizes, data sparseness, and degree of separation. Further, noise in the data can... |

66 | Distribution free decomposition of multivariate data
- Comaniciu, Meer
- 1999
(Show Context)
Citation Context ...ers, merging being based on inter-cluster similarity; a final partition is selected among the clustering hierarchy by thresholding techniques or based on measures of cluster validity [39], [5], [40], =-=[41]-=-, [42], [43]. Treating the clustering problem as a graph partitioning problem, a recent approach, known as spectral clustering, applies spectral graph theory for clustering [44], [45], [46]. Among the... |

52 |
Information theoretic clustering
- Gokcay, Principe
(Show Context)
Citation Context ...rwise clustering cost function, emphasizing cluster connectedness. Non-parametric density based clustering methods attempt to identify high density clusters separated by low density regions [5] [32], =-=[33]-=-. Graph-theoretical approaches [34] have mostly been explored in hierarchical methods, that can be represented graphically as a tree or dendrogram [7], [8]. Both agglomerative [28], [35] and divisive ... |

43 | Large-scale parallel data clustering
- JUDD, MCKINLEY, et al.
- 1996
(Show Context)
Citation Context ...tion a set of unlabelled objects into homogeneous groups or clusters. A number of application areas use clustering techniques for organizing or discovering structure in data, such as data mining [1], =-=[2]-=-, information retrieval [3], [4], [5], image segmentation [6], and machine learning. In real world problems, clusters can appear with different shapes, sizes, data sparseness, and degree of separation... |

35 |
Validity studies in clustering methodologies
- Dubes, Jain
- 1976
(Show Context)
Citation Context ...encountered in practice. Each algorithm has its own approach for estimating the number of clusters [12], [13], imposing a structure on the data [14], [15], [16], and validating the resulting clusters =-=[17]-=-, [18], [19], [20], [21], [22]. Model-based techniques assume particular cluster shapes that can be given a simple and compact description. Examples of model-based techniques include: parametric densi... |

32 |
Ensemble methods in machine learning," in Multiple Classifier Systems
- Dietterich
- 2000
(Show Context)
Citation Context ...t is wise to apply several different clustering algorithms to the given data and then determine the best algorithm for the data. Inspired by the work in sensor fusion and classifier combination [50], =-=[51]-=-, [52], a clustering combination approach has been proposed [53], [54], [55]. Fred and Jain introduce the concept of evidence accumulation clustering, that maps the individual data partitions in a clu... |

32 |
Classifier combinations: Implementations and theoretical issues
- Lam
- 2000
(Show Context)
Citation Context ...ise to apply several different clustering algorithms to the given data and then determine the best algorithm for the data. Inspired by the work in sensor fusion and classifier combination [50], [51], =-=[52]-=-, a clustering combination approach has been proposed [53], [54], [55]. Fred and Jain introduce the concept of evidence accumulation clustering, that maps the individual data partitions in a clusterin... |

30 |
A New Shared Nearest Neighbor Clustering Algorithm and its Applications
- Ertöz, Steinbach, et al.
- 2002
(Show Context)
Citation Context ...troid is seen as a prototype of a region in the feature space. The next phase is to remove noisy patterns based on density analysis. We build on the ideas of Jarvis and Patrick [60] and Ertoz- et al. =-=[61]-=- to obtain a set of “core”points from the available data. Jarvis and Patrick defined a shared nearest neighbor graph by the process of k-nearest neighbor sparsification: a link between patterns i and ... |

26 |
On finding the number of clusters
- Kothari, Pitts
- 1999
(Show Context)
Citation Context ...et no single algorithm is able to identify all sorts of cluster shapes and structures that are encountered in practice. Each algorithm has its own approach for estimating the number of clusters [12], =-=[13]-=-, imposing a structure on the data [14], [15], [16], and validating the resulting clusters [17], [18], [19], [20], [21], [22]. Model-based techniques assume particular cluster shapes that can be given... |

24 |
Bootstrap Technique in Cluster Analysis
- Jain, Moreau
- 1987
(Show Context)
Citation Context ...11], yet no single algorithm is able to identify all sorts of cluster shapes and structures that are encountered in practice. Each algorithm has its own approach for estimating the number of clusters =-=[12]-=-, [13], imposing a structure on the data [14], [15], [16], and validating the resulting clusters [17], [18], [19], [20], [21], [22]. Model-based techniques assume particular cluster shapes that can be... |

24 |
Spectral kernel methods for clustering
- Cristianini, Taylor, et al.
- 2001
(Show Context)
Citation Context ...[5], [40], [41], [42], [43]. Treating the clustering problem as a graph partitioning problem, a recent approach, known as spectral clustering, applies spectral graph theory for clustering [44], [45], =-=[46]-=-. Among the various clustering methods, the K-means algorithm, which minimizes the squared-error criteria, is one of the simplest clustering algorithm. It is computationally efficient and does not req... |

19 | Path based pairwise data clustering with application to texture segmentation. In: Energy minimization methods in computer vision and pattern recognition, LNCS
- Fischer, Zöller, et al.
- 2001
(Show Context)
Citation Context ...rated in the clustering procedure [29], [30], [25]. Most of the above techniques utilize an optimization procedure tuned to a particular cluster shape, or emphasize cluster compactness. Fisher et al. =-=[31]-=- proposed an optimization-based clustering algorithm, based on a pairwise clustering cost function, emphasizing cluster connectedness. Non-parametric density based clustering methods attempt to identi... |

19 | Location- and density-based hierarchical clustering using similarity analysis
- BAJCSY, AHUJA
- 1980
(Show Context)
Citation Context ... being based on inter-cluster similarity; a final partition is selected among the clustering hierarchy by thresholding techniques or based on measures of cluster validity [39], [5], [40], [41], [42], =-=[43]-=-. Treating the clustering problem as a graph partitioning problem, a recent approach, known as spectral clustering, applies spectral graph theory for clustering [44], [45], [46]. Among the various clu... |

18 | Principal curve clustering with noise
- Stanford, Raftery
- 1997
(Show Context)
Citation Context ...l sorts of cluster shapes and structures that are encountered in practice. Each algorithm has its own approach for estimating the number of clusters [12], [13], imposing a structure on the data [14], =-=[15]-=-, [16], and validating the resulting clusters [17], [18], [19], [20], [21], [22]. Model-based techniques assume particular cluster shapes that can be given a simple and compact description. Examples o... |

17 |
Detection and separation of ring-shaped clusters using fuzzy clustering
- Man, Gath
- 1994
(Show Context)
Citation Context ...s of cluster shapes and structures that are encountered in practice. Each algorithm has its own approach for estimating the number of clusters [12], [13], imposing a structure on the data [14], [15], =-=[16]-=-, and validating the resulting clusters [17], [18], [19], [20], [21], [22]. Model-based techniques assume particular cluster shapes that can be given a simple and compact description. Examples of mode... |

16 | Concept learning and feature selection based on squareerror clustering
- Mirkin
- 1999
(Show Context)
Citation Context ...d techniques include: parametric density approaches, such as mixture decomposition techniques [23], [24], [25], [26]; prototype-based methods, such as central clustering [14], square-error clustering =-=[27]-=-, K-means [28], [8] or K-medoids clustering [9]; and shape fitting approaches [15], [6], [16]. Model order selection is sometimes left as a design parameter to be specified by the user, or it is incor... |

16 | An improved branch and bound algorithm for computing k -nearest neighbors - KAMGAR-PARSI, KANAL - 1985 |

15 | Evidence accumulation clustering based on the k-means algorithm
- Fred, Jain
(Show Context)
Citation Context ... and then determine the best algorithm for the data. Inspired by the work in sensor fusion and classifier combination [50], [51], [52], a clustering combination approach has been proposed [53], [54], =-=[55]-=-. Fred and Jain introduce the concept of evidence accumulation clustering, that maps the individual data partitions in a clustering ensemble into a new similarity measure between patterns, summarizing... |

14 | Conceptual clustering in information retrieval
- Bhatia, Deogun
- 1998
(Show Context)
Citation Context ...jects into homogeneous groups or clusters. A number of application areas use clustering techniques for organizing or discovering structure in data, such as data mining [1], [2], information retrieval =-=[3]-=-, [4], [5], image segmentation [6], and machine learning. In real world problems, clusters can appear with different shapes, sizes, data sparseness, and degree of separation. Further, noise in the dat... |

13 | A monothetic clustering method
- CHAVENT
- 1999
(Show Context)
Citation Context ...oretical approaches [34] have mostly been explored in hierarchical methods, that can be represented graphically as a tree or dendrogram [7], [8]. Both agglomerative [28], [35] and divisive approaches =-=[36]-=- (such as those based on the minimum spanning tree - MST [28]) have been proposed; different algorithms are obtained depending on the definition of similarity measures between patterns and between clu... |

10 |
On-line hierarchical clustering
- El-Sonbaty, Ismail
- 1998
(Show Context)
Citation Context ...ons [5] [32], [33]. Graph-theoretical approaches [34] have mostly been explored in hierarchical methods, that can be represented graphically as a tree or dendrogram [7], [8]. Both agglomerative [28], =-=[35]-=- and divisive approaches [36] (such as those based on the minimum spanning tree - MST [28]) have been proposed; different algorithms are obtained depending on the definition of similarity measures bet... |

9 | Clustering under a hypothesis of smooth dissimilarity increments - Fred, Leitão |

9 | Mdl-based selection of the number of components in mixture models for pattern recognition
- Tenmoto, Kudo, et al.
- 1998
(Show Context)
Citation Context ...stering [9]; and shape fitting approaches [15], [6], [16]. Model order selection is sometimes left as a design parameter to be specified by the user, or it is incorporated in the clustering procedure =-=[29]-=-, [30], [25]. Most of the above techniques utilize an optimization procedure tuned to a particular cluster shape, or emphasize cluster compactness. Fisher et al. [31] proposed an optimization-based cl... |

8 |
Cluster validity profiles
- Bailey, Dubes
- 1982
(Show Context)
Citation Context ...tered in practice. Each algorithm has its own approach for estimating the number of clusters [12], [13], imposing a structure on the data [14], [15], [16], and validating the resulting clusters [17], =-=[18]-=-, [19], [20], [21], [22]. Model-based techniques assume particular cluster shapes that can be given a simple and compact description. Examples of model-based techniques include: parametric density app... |

8 |
Probabilistic validation approach for clustering
- Har-Even, Brailovsky
- 1995
(Show Context)
Citation Context ...in practice. Each algorithm has its own approach for estimating the number of clusters [12], [13], imposing a structure on the data [14], [15], [16], and validating the resulting clusters [17], [18], =-=[19]-=-, [20], [21], [22]. Model-based techniques assume particular cluster shapes that can be given a simple and compact description. Examples of model-based techniques include: parametric density approache... |

6 |
Finding regions of interest for content-extraction
- Pauwels, Frederix
- 1999
(Show Context)
Citation Context ... homogeneous groups or clusters. A number of application areas use clustering techniques for organizing or discovering structure in data, such as data mining [1], [2], information retrieval [3], [4], =-=[5]-=-, image segmentation [6], and machine learning. In real world problems, clusters can appear with different shapes, sizes, data sparseness, and degree of separation. Further, noise in the data can mask... |