Abstract

Knowledge-based natural language processing systems have achieved good success with certain tasks but they are often criticized because they depend on a domain-specific dictionary that requires a great deal of manual knowledge engineering. This knowledge engineering bottleneck makes knowledge-based NLP systems impractical for real-world applications because they cannot be easily scaled up or ported to new domains. In response to this problem, we developed a system called AutoSlog that automatically builds a domain-specific dictionary of concepts for extracting information from text. Using AutoSlog, we constructed a dictionary for the domain of terrorist event descriptions in only 5 person-hours. We then compared the AutoSlog dictionary with a hand-crafted dictionary that was built by two highly skilled graduate students and required approximately 1500 person-hours of effort. We evaluated the two dictionaries using two blind test sets of 100 texts each. Overall, the AutoSlog dictionary achieved 98% of the performance of the hand-crafted dictionary. On the first test set, the AutoSlog dictionary obtained 96.3% of the performance of the hand-crafted dictionary. On the second test set, the overall scores were virtually indistinguishable with the AutoSlog dictionary achieving 99.7% of the performance of the handcrafted dictionary.

Citations

333	Explanation-Based Learning: An Alternative View - DeJong, Mooney - 1986
168	Frequency Analysis of English Usage - Francis, Kucera - 1982
81	A performance evaluation of text-analysis technologies - Lehnert, Sundheim - 1991
67	FOUL-UP: A Program that Figures Out Meanings of Words from Context - Granger - 1977
45	Symbolic/subsymbolic sentence analysis: Exploiting the best of two worlds - Lehnert - 1991
41	Explanation-Based Generalization: A Unifying View. Machine Learning 1:1 - Mitchel, Keller, et al. - 1986
36	Acquiring Lexical Knowledge from Text: A Case Study - Jacobs, Zernik - 1988
24	Towards a Self-Extending Parser - Carbonell - 1979
23	Learning to disambiguate relative pronouns - Cardie - 1992
21	ÒAutomated Dictionary Construction for Information Extraction from Text,Ó - Riloff, Lehnert - 1993
16	ÒUniversity of Massachusetts: MUC-4 Test Results and Analysis,Ó - Lehnert, Fisher, et al. - 1992