Abstract

Keyphrases are useful for a variety of purposes, including summarizing, indexing, labeling, categorizing, clustering, highlighting, browsing, and searching. The task of automatic keyphrase extraction is to select keyphrases from within the text of a given document. Automatic keyphrase extraction makes it feasible to generate keyphrases for the huge number of documents that do not have manually assigned keyphrases. A limitation of previous keyphrase extraction algorithms is that the selected keyphrases are occasionally incoherent. That is, the majority of the output keyphrases may fit together well, but there may be a minority that appear to be outliers, with no clear semantic relation to the majority or to each other. This paper presents enhancements to the Kea keyphrase extraction algorithm that are designed to increase the coherence of the extracted keyphrases. The approach is to use the degree of statistical association among candidate keyphrases as evidence that they may be semantically related. The statistical association is measured using web mining. Experiments demonstrate that the enhancements improve the quality of the extracted keyphrases. Furthermore, the enhancements are not domain-specific: the algorithm generalizes well when it is trained on one domain (computer science documents) and tested on another (physics documents). 1

Citations

627	Word association norms, mutual information, and lexicography - Church, Hanks - 1990
531	Multi-interval discretization of continuousvalued attributes for classification learning - Fayyad, Irani - 1993
486	Pazzani M: On the optimality of the simple Bayesian classifier under zero-one loss - Domingos - 1997
419	Inductive learning algorithms and representations for text categorization - Dumais, Platt, et al. - 1998
276	Using lexical chains for text summarization - Barzilay, Elhadad - 1997
269	Lexical cohesion computed by thesaural relations as an indicator of the structure of text - Morris, Hirst - 1991
131	Using statistics in lexical analysis - Church, Gale, et al. - 1991
118	Mining the web for synonyms: Pmi-ir versus lsa on toefl - Turney
116	Domain-specific keyphrase extraction - Frank, Paynter, et al. - 1999
94	Learning Algorithms for Keyphrase Extraction - Turney - 2000
88	Information Retrieval. 2nd edition - Rijsbergen - 1979
70	KEA: Practical Automatic Keyphrase Extraction - Witten, Paynter, et al. - 1999
49	Improving browsing in digital libraries with keyphrase indexes - Gutwin, Paynter, et al. - 2003
39	Learning to Extract Keyphrases from Text - Turney - 1999
14	Automatic extraction of document keyphrases for use in digital libraries: evaluation and applications - Jones, Paynter
13	Which Method Learns the Most from Data? Methodological Issues in the Analysis of Comparative Studies - Feelders, Verkooijen - 1995
10	Human evaluation of kea, an automatic keyphrasing system - Jones, Paynter - 2001
10	A statistical learning approach to automatic indexing of controlled index terms - Leung, Kan - 1997
1	Mining the Web for synonyms: PMI-1R versus LSA on TOEFL - Turney - 2001