DMCA
Latent Dirichlet Allocation in Web Spam Filtering. manuscript (2008)
| Citations: | 24 - 5 self |
BibTeX
@MISC{Bíró08latentdirichlet,
author = {István Bíró and Dávid Siklósi and Jácint Szabó and András A. Benczúr},
title = {Latent Dirichlet Allocation in Web Spam Filtering. manuscript},
year = {2008}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Latent Dirichlet allocation (LDA) (Blei, Ng, Jordan 2003) is a fully generative statistical language model on the content and topics of a corpus of documents. In this paper we apply an extension of LDA for web spam classification. Our linked LDA technique takes also linkage into account: topics are propagated along links in such a way that the linked document directly influences the words in the linking document. The inferred LDA model can be applied for classification as dimensionality reduction similarly to latent semantic indexing. We test linked LDA on the WEBSPAM-UK2007 corpus. By using BayesNet classifier, in terms of the AUC of classification, we achieve 3 % improvement over plain LDA with BayesNet, and 8 % over the public link features with C4.5. The addition of this method to a log-odds based combination of strong link and content baseline classifiers results in a 3 % improvement in AUC. Our method even slightly improves over the best Web Spam Challenge 2008 result.
Keyphrases
latent dirichlet allocation web spam filtering web spam challenge linking document web spam classification semantic indexing linked lda technique inferred lda model bayesnet classifier public link feature content baseline classifier result strong link webspam-uk2007 corpus generative statistical language model plain lda dimensionality reduction
