Abstract

We investigate prototype-driven learning for primarily unsupervised sequence modeling. Prior knowledge is specified declaratively, by providing a few canonical examples of each target annotation label. This sparse prototype information is then propagated across a corpus using distributional similarity features in a log-linear generative model. On part-of-speech induction in English and Chinese, as well as an information extraction task, prototype features provide substantial error rate reductions over competitive baselines and outperform previous work. For example, we can achieve an English part-of-speech tagging accuracy of 80.5 % using only three examples of each tag and no dictionary constraints. We also compare to semi-supervised learning and discuss the system’s error trends. 1

Citations

3118	A tutorial on hidden Markov models and selected applications in speech recognition - Rabiner - 1989
2168	Indexing by Latent Semantic Analysis - Deerwester, Dumais, et al. - 1990
1655	M: Building a Large Annotated Corpus of English: The Penn Tree Bank - Marcus, Santorini, et al. - 1993
1548	F: Conditional Random Fields: Probabilistic models for segmenting and labeling sequence data - Lafferty, McCallum, et al. - 2001
312	On the limited memory BFGS method for large scale optimization - Liu, Nocedal - 1989
212	Tagging English text with probabilistic model - Merialdo - 1994
75	Distributional Part-of-Speech Tagging - Schütze - 1994
49	Transformational grammar - Radford - 1988
28	Unsupervised learning of field segmentation models for information extraction - Grenager, Klein, et al. - 2005