Abstract

We present conditional random fields, a framework for building probabilistic models to segment and label sequence data. Conditional random fields offer several advantages over hidden Markov models and stochastic grammars for such tasks, including the ability to relax strong independence assumptions made in those models. Conditional random fields also avoid a fundamental limitation of maximum entropy Markov models (MEMMs) and other discriminative Markov models based on directed graphical models, which can be biased towards states with few successor states. We present iterative parameter estimation algorithms for conditional random fields and compare the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data. 1.

Citations

1714	Schapire R: A decision-theoretic generalization of online learning and an application to boosting - Freund - 1997
846	A Maximum Entropy Approach to Natural Language Processing - Berger, Pietra, et al. - 1996
662	Transformation-Based Error-Driven Learning and Natural Language Processing A Case Study - Brill - 1995
598	Biological sequence analysis: probabilistic models of proteins and nucleic acids. Cambridge Univ - Durbin, Eddy, et al. - 2000
521	Foundations of statistical natural language processing - Manning, Schütze - 1999
487	Gradient-based learning applied to document recognition - LeCun, Bottou, et al. - 1998
464	Inducing Features of Random Fields - Pietra, Pietra, et al. - 1997
355	Generalized Iterative Scaling for Log-Linear Models - Darroch, Ratcliff - 1972
355	Maximum entropy Markov models for information extraction and segmentation - McCallum, Freitag, et al. - 2000
348	A maximum entropy model for part-of-speech tagging - Ratnaparkhi - 1996
260	Finite-state transducers in language and speech processing - Mohri - 1997
220	Discriminative Reranking for Natural Language Parsing - Collins - 2000
154	Learning to resolve natural language ambiguities: A unified approach - Roth - 1998
131	Introduction to Probabilistic Automata - Paz - 1971
89	Information Extraction with HMM Structures Learned by Stochastic Optimization - Freitag, McCallum - 2000
84	Markov fields on finite graphs and lattices,” Unpublished manuscript - Hammersley, Clifford - 1971
62	Boosting applied to tagging and PP attachment - Abney, Schapire, et al. - 1999
47	Une approche theorique de l'apprentissage connexioniste� applications a la reconnaissance de la parole - Bottou - 1991
47	Minimization Algorithms for Sequential Transducers - Mohri
47	Boltzmann chains and hidden Markov models - Saul, Jordan - 1996
25	A whole sentence maximum entropy language model - Rosenfeld - 1997
7	Equivalence of linear boltzmann chains and hidden markov models - MacKay - 1996
3	The use of classifiers in sequential inference. NIPS 13. Forthcoming - Punyakanok, Roth - 2001