Clustering with Bregman Divergences

DMCA

Clustering with Bregman Divergences (2005)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Arindam Banerjee , Srujana Merugu , Inderjit Dhillon , Joydeep Ghosh

Venue:	JOURNAL OF MACHINE LEARNING RESEARCH
Citations:	441 - 57 self

BibTeX

@INPROCEEDINGS{Banerjee05clusteringwith,
    author = {Arindam Banerjee and Srujana Merugu and Inderjit Dhillon and Joydeep Ghosh},
    title = {Clustering with Bregman Divergences},
    booktitle = {JOURNAL OF MACHINE LEARNING RESEARCH},
    year = {2005},
    pages = {1705--1749},
    publisher = {JMLR.org}
}

OpenURL

Abstract

A wide variety of distortion functions are used for clustering, e.g., squared Euclidean distance, Mahalanobis distance and relative entropy. In this paper, we propose and analyze parametric hard and soft clustering algorithms based on a large class of distortion functions known as Bregman divergences. The proposed algorithms unify centroid-based parametric clustering approaches, such as classical kmeans and information-theoretic clustering, which arise by special choices of the Bregman divergence. The algorithms maintain the simplicity and scalability of the classical kmeans algorithm, while generalizing the basic idea to a very large class of clustering loss functions. There are two main contributions in this paper. First, we pose the hard clustering problem in terms of minimizing the loss in Bregman information, a quantity motivated by rate-distortion theory, and present an algorithm to minimize this loss. Secondly, we show an explicit bijection between Bregman divergences and exponential families. The bijection enables the development of an alternative interpretation of an ecient EM scheme for learning models involving mixtures of exponential distributions. This leads to a simple soft clustering algorithm for all Bregman divergences.

Keyphrases

bregman divergence distortion function classical kmeans large class rate-distortion theory loss function hard clustering problem relative entropy exponential distribution wide variety alternative interpretation exponential family euclidean distance basic idea explicit bijection special choice bregman information main contribution ecient em scheme mahalanobis distance information-theoretic clustering

DMCA