Results 1 
1 of
1
A probabilistic model of redundancy in information extraction
 IN IJCAI
, 2005
"... Unsupervised Information Extraction (UIE) is the task of extracting knowledge from text without using handtagged training examples. A fundamental problem for both UIE and supervised IE is assessing the probability that extracted information is correct. In massive corpora such as the Web, the same e ..."
Abstract

Cited by 95 (18 self)
 Add to MetaCart
Unsupervised Information Extraction (UIE) is the task of extracting knowledge from text without using handtagged training examples. A fundamental problem for both UIE and supervised IE is assessing the probability that extracted information is correct. In massive corpora such as the Web, the same extraction is found repeatedly in different documents. How does this redundancy impact the probability of correctness? This paper introduces a combinatorial “ballsandurns” model that computes the impact of sample size, redundancy, and corroboration from multiple distinct extraction rules on the probability that an extraction is correct. We describe methods for estimating the model’s parameters in practice and demonstrate experimentally that for UIE the model’s log likelihoods are 15 times better, on average, than those obtained by Pointwise Mutual Information (PMI) and the noisyor model used in previous work. For supervised IE, the model’s performance is comparable to that of Support Vector Machines, and Logistic Regression.