Abstract

Many academic journals ask their authors to provide a list of about five to fifteen keywords, to appear on the first page of each article. Since these key words are often phrases of two or more words, we prefer to call them keyphrases. There is a wide variety of tasks for which keyphrases are useful, as we discuss in this paper. We approach the problem of automatically extracting keyphrases from text as a supervised learning task. We treat a document as a set of phrases, which the learning algorithm must learn to classify as positive or negative examples of keyphrases. Our first set of experiments applies the C4.5 decision tree induction algorithm to this learning task. We evaluate the performance of nine different configurations of C4.5. The second set of experiments applies the GenEx algorithm to the task. We developed the GenEx algorithm specifically for automatically extracting keyphrases from text. The experimental results support the claim that a custom-designed algorithm (GenEx)...

Citations

4071	C4.5: Programs for machine learning - Quinlan - 1993
2888	Induction of Decision Trees - Quinlan - 1986
1998	Bagging predictors - Breiman - 1996
1459	An algorithm for suffix stripping - Porter - 1980
1325	Experiments with a new boosting algorithm - Freund, Schapire - 1996
443	The Automatic Creation of Literature Abstracts. IBM Journal of Research and Development - Luhn - 1958
342	A trainable document summarizer - Kupiec, Pedersen, et al. - 1995
330	Optimization of Control Parameters for Genetic Algorithms - Grefenstette - 1986
251	Bagging, boosting, and C4.5 - Quinlan - 1996
236	Viewing morphology as an inference process - Krovetz - 1993
232	Arcing classifiers - Breiman - 1998
190	New Methods in Automatic Extracting - Edmundson - 1969
184	Development of a stemming algorithm - Lovins - 1968
163	Constructing literature abstracts by computer: techniques and prospects - Paice - 1990
146	Experiments in automatic phrase indexing for document retrieval: A comparison of syntactic and non-syntactic methods - Fagan - 1987
116	D.D.: The use of phrases and structured queries in information retrieval - Croft, Turtle, et al. - 1991
110	Automatic condensation of electronic publications by sentence selection - Brandow, Mitze, et al. - 1995
80	Megainduction: Machine Learning on Very large Databases - Catlett - 1991
80	An empirical evaluation of bagging and boosting - Maclin, Opitz - 1997
75	Automatic analysis, theme generation, and summarization of machine-readable text, Science 264 - Salton, Allan, et al. - 1994
59	The identification of important concepts in highly structured technical papers - Paice, Jones - 1993
49	Improving browsing in digital libraries with keyphrase indexes - Gutwin, Paynter, et al. - 2003
49	Learning User Information Interests Through Extraction of Semantically Significant Phrases - Krulwich, Burkley - 1996
39	Learning to Extract Keyphrases from Text - Turney - 1999
33	Decision trees as probabilistic classifiers - Quinlan - 1987
29	Wrap-up: a trainable discourse module for information extraction - Soderland, Lehnert - 1994
28	Assessing credit card applications using machine learning - Carter, Catlett - 1987
28	A user's guide to GENESIS - Grefenstette - 1984
24	The application of linguistic processing to automatic abstract generation." J Documentation and Text Management 1 - Johnson, Paice, et al. - 1993
22	Extraction of Keyphrases from Text: Evaluation of Four Algorithms - Turney - 1997
19	Towards automatic indexing: automatic assignment of controlled-language indexing and classification from free indexing - Field - 1975
17	Exporting phrases: A statistical analysis of topical language - Steier, Belew - 1993
14	A Unified Approach to Automatic Indexing and Information Retrieval - Ginsberg - 1993
13	Which Method Learns the Most from Data? Methodological Issues in the Analysis of Comparative Studies - Feelders, Verkooijen - 1995
11	A production rule system for message summarization - Marsh, Hamburger, et al. - 1984
11	A Syntactic Approach to Automatic Book Indexing - Salton - 1989
10	A statistical learning approach to automatic indexing of controlled index terms - Leung, Kan - 1997
8	Development of a document summarization system for effective information services - Jang, Myaeng - 1997
7	Probability and statistics: Theory and applications - Fraser - 1976
5	Stratifying samples to improve learning - Buntine - 1989
5	Extraction of index words from manuals - Nakagawa - 1997
5	Does indexing exhaustivity matter - Jones, K - 1973
3	Domain-specific Learning Algorithms for Keyphrase Extraction keyphrase extraction - Frank, Paynter, et al. - 1999
2	Sample surveys: The field - Deming - 1978
2	Adaptation of a keyphrase extractor for Japanese text - Mathieu - 1999
2	Compound key word generation from document databases using a hierarchical clustering ART model - Muoz - 1996
1	SRA: Description of the SRA system as used for MUC-6. Proceedings of the Sixth Message Understanding Conference - Krupka - 1995