Abstract

Since their recent introduction, conditional random fields (CRFs) have been success-fully applied to a multitude of structured labelling tasks in many different domains. Examples include natural language processing (NLP), bioinformatics and computer vision. Within NLP itself we have seen many different application areas, like named entity recognition, shallow parsing, information extraction from research papers and language modelling. Most of this work has demonstrated the need, directly or indi-rectly, to employ some form of regularisation when applying CRFs in order to over-come the tendency for these models to overfit. To date a popular method for regularis-ing CRFs has been to fit a Gaussian prior distribution over the model parameters. In this thesis we explore other methods of CRF regularisation, investigating their properties and comparing their effectiveness. We apply our ideas to sequence labelling problems in NLP, specifically part-of-speech tagging and named entity recognition. We start with an analysis of conventional approaches to CRF regularisation, and investigate possible extensions to such approaches. In particular, we consider choices

Citations

1548	BConditional random fields: Probabilistic models for segmenting and labeling sequence data - Lafferty, McCallum, et al.
464	Inducing Features of Random Fields - Pietra, Pietra, et al. - 1997
336	F: Shallow parsing with conditional random fields - Sha, Pereira
171	A comparison of algorithms for maximum entropy parameter estimation - Malouf - 2002
156	Some statistical issues in the comparison of speech recognition algorithms - Gillick, Cox - 1989
142	Efficiently inducing features or conditional random fields - McCallum - 2003
116	Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons - McCallum, Li - 2003
112	Products of experts - Hinton - 1999
105	S: Introduction to the CoNLL-2000 shared task: chunking - Sang, Buchholz
96	Accurate information extraction from research papers using conditional random fields - Peng, McCallum - 2004
65	An Integrated, Conditional Model of Information Extraction and Coreference with Application to Citation Matching - Wellner, McCallum, et al. - 2004
65	Language independent NER using a maximum entropy tagger - JR, Clark
35	Bayesian conditional random fields - Qi, Szummer, et al. - 2005
26	Using diversity in preparing ensembles of classifiers based on different feature subsets to minimize generalization error - Zenobi, Cunningham - 2001
25	More accurate tests for the statistical significance of result differences - Yeh
19	Selecting weighting factors in logarithmic opinion pools - Heskes - 1998
18	A multiplicative formula for aggregating probability assessments - Bordley - 1982
18	Dynamic Conditional Random Fields for jointly labeling multiple sequences - McCallum, Rohanimanesh, et al. - 2003
18	Ensemblebased active learning for parse selection - Osborne, Baldridge - 2004
12	Bayesian regularisation and pruning using a laplace prior - Williams - 1995
10	Scaling Conditional Random Fields Using Error-Correcting Codes - Cohn, Smith, et al.