Type Level Clustering Evaluation: New Measures and a POS Induction Case Study

by Roi Reichart , Omri Abend

Citations:

3 - 0 self

Datum	Value	Source
TITLE	Type Level Clustering Evaluation: New Measures and a POS Induction Case Study	SVM HeaderParse 0.2
AUTHOR NAME	Roi Reichart	SVM HeaderParse 0.2
AUTHOR AFFIL	1; ICNC; 2; Institute of Computer Science; Hebrew University of Jerusalem	SVM HeaderParse 0.2
AUTHOR NAME	Omri Abend	SVM HeaderParse 0.2
AUTHOR AFFIL	1; ICNC; 2; Institute of Computer Science; Hebrew University of Jerusalem	SVM HeaderParse 0.2
ABSTRACT	Clustering is a central technique in NLP. Consequently, clustering evaluation is of great importance. Many clustering algorithms are evaluated by their success in tagging corpus tokens. In this paper we discuss type level evaluation, which reflects class membership only and is independent of the token statistics of a particular reference corpus. Type level evaluation casts light on the merits of algorithms, and for some applications is a more natural measure of the algorithm’s quality. We propose new type level evaluation measures that, contrary to existing measures, are applicable when items are polysemous, the common case in NLP. We demonstrate the benefits of our measures using a detailed case study, POS induction. We experiment with seven leading algorithms, obtaining useful insights and showing that token and type level measures can weakly or even negatively correlate, which underscores the fact that these two approaches reveal different aspects of clustering quality. 1	SVM HeaderParse 0.2
CITATIONS	35 found	ParsCit 1.0