Abstract

In many supervised learning tasks, the entities to be labeled are related to each other in complex ways and their labels are not independent. For example, in hypertext classification, the labels of linked pages are highly correlated. A standard approach is to classify each entity independently, ignoring the correlations between them. Recently, Probabilistic Relational Models, a relational version of Bayesian networks, were used to define a joint probabilistic model for a collection of related entities. In this paper, we present an alternative framework that builds on (conditional) Markov networks and addresses two limitations of the previous approach. First, undirected models do not impose the acyclicity constraint that hinders representation of many important relational dependencies in directed models. Second, undirected models are well suited for discriminative training, where we optimize the conditional likelihood of the labels given the features, which generally improves classification accuracy. We show how to train these models effectively, and how to use approximate probabilistic inference over the learned model for collective classification of multiple related entities. We provide experimental results on a webpage classification task, showing that accuracy can be significantly improved by modeling relational dependencies. 1

Citations

5662	Probabilistic reasoning in intelligent systems: networks of plausible inference - Pearl - 1988
1548	F: Conditional Random Fields: Probabilistic models for segmenting and labeling sequence data - Lafferty, McCallum, et al. - 2001
464	Inducing features of random fields - Pietra, S, et al. - 1997
421	Learning probabilistic relational models - Friedman, Getoor, et al. - 1999
364	Loopy belief propagation for approximate inference: an empirical study - Murphy, Weiss, et al. - 1999
326	Enhanced hypertext categorization using hyperlinks - Chakrabarti, Dom, et al. - 1998
290	DiPasquo,“Learning to extract Symbolic Knowledge from the World Wide Web - Craven, Freitag, et al. - 1998
168	Probabilistic frame-based systems - Koller, Pfeffer - 1998
98	Iterative classification in relational data - Neville, Jensen - 2000
84	Probabilistic classification and clustering in relational data - Taskar, Segal, et al. - 2001
78	A study of approaches to hypertext categorization - Yang, Slattery, et al. - 2002
48	Discovering test set regularities in relational domains - Slattery, Mitchell - 2000
47	Introduction to Informetrics - Egghe, Rousseau - 1990
45	Algorithms for maximum-likelihood logistic regression - Minka - 2001
1	Authoritative sources in a hyperlinked environment - Publishers, Francisco, et al. - 1999