Larger Residuals, Less Work: Active Document Scheduling for Latent Dirichlet Allocation

Larger Residuals, Less Work: Active Document Scheduling for Latent Dirichlet Allocation

Cached

by Mirwaes Wahabzada , Kristian Kersting

Citations:

1 - 1 self

Versions

Version 0

Version History

Metadata Version 0

Datum	Value	Source
TITLE	Larger Residuals, Less Work: Active Document Scheduling for Latent Dirichlet Allocation	SVM HeaderParse 0.2
AUTHOR NAME	Mirwaes Wahabzada	SVM HeaderParse 0.2
AUTHOR AFFIL	Knowledge Discovery Department, Fraunhofer IAIS	SVM HeaderParse 0.2
AUTHOR ADDR	Schloss Birlinghoven, 53754 Sankt Augustin, Germany	SVM HeaderParse 0.2
AUTHOR NAME	Kristian Kersting	SVM HeaderParse 0.2
AUTHOR AFFIL	Knowledge Discovery Department, Fraunhofer IAIS	SVM HeaderParse 0.2
AUTHOR ADDR	Schloss Birlinghoven, 53754 Sankt Augustin, Germany	SVM HeaderParse 0.2
ABSTRACT	Abstract. Recently, there have been considerable advances in fast inference for latent Dirichlet allocation (LDA). In particular, stochastic optimization of the variational Bayes (VB) objective function with a natural gradient step was proved to converge and able to process massive document collections. To reduce noise in the gradient estimation, it considers multiple documents chosen uniformly at random. While it is widely recognized that the scheduling of documents in stochastic optimization may have significant consequences, this issue remains largely unexplored. In this work, we address this issue. Specifically, we propose residual LDA, a novel, easy-to-implement, LDA approach that schedules documents in an informed way. Intuitively, in each iteration, residual LDA actively selects documents that exert a disproportionately large influence on the current residual to compute the next update. On several real-world datasets, including 3M articles from Wikipedia, we demonstrate that residual LDA can handily analyze massive document collections and find topic models as good or better than those found with batch VB and randomly scheduled VB, and significantly faster. 1	SVM HeaderParse 0.2
CITATIONS	23 found	ParsCit 1.0