Abstract

We consider the problem of improving named entity recognition (NER) systems by using external dictionaries---more specifically, the problem of extending state-of-the-art NER systems by incorporating information about the similarity of extracted entities to entities in an external dictionary. This is di#cult because most high-performance named entity recognition systems operate by sequentially classifying words as to whether or not they participate in an entity name; however, the most useful similarity measures score entire candidate names. To correct this mismatch we formalize a semi-Markov extraction process, which is based on sequentially classifying segments of several adjacent words, rather than single words. In addition to allowing a natural way of coupling high-performance NER methods and highperformance similarity functions, this formalism also allows the direct use of other useful entity-level features, and provides a more natural formulation of the NER problem than sequential word classification. Experiments in multiple domains show that the new model can substantially improve extraction performance over previous methods for using external dictionaries in NER.

Citations

275	Snowball: Extracting relations from large plain-text collections - Agichtein, Gravano - 2000
270	An Algorithm that Learns What’s in a Name - Bikel, Schwartz, et al. - 1999
154	Hidden markov support vector machines - Altun, Tsochantaridis, et al. - 2003
89	Exploiting Diverse Knowledge Sources via Maximum Entropy in Named Entity Recognition - Borthwick, Sterling, et al. - 1998
11	Automatic text segmentation for extracting structured records - Borkar, Deshmukh, et al. - 2001
10	Learning to extract proteins and their interactions from medline abstracts. Available from http://www.cs.utexas.edu/users/ml/publication/ie.html - Bunescu, Ge, et al. - 2002
9	Extracting gene and protein names from biomedical abstracts. Unpublished Technical Note, Available from http://www.cs.utexas.edu/users/ml/publication/ie.html - Bunescu, Ge, et al. - 2004