DMCA
Distributed Representations of Sentences and Documents
Cached
Download Links
| Venue: | In NAACL HLT |
| Citations: | 93 - 1 self |
BibTeX
@INPROCEEDINGS{Le_distributedrepresentations,
author = {Quoc Le and Tomas Mikolov},
title = {Distributed Representations of Sentences and Documents},
booktitle = {In NAACL HLT},
year = {},
pages = {2013}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Many machine learning algorithms require the input to be represented as a fixed-length feature vector. When it comes to texts, one of the most common fixed-length features is bag-of-words. Despite their popularity, bag-of-words features have two major weaknesses: they lose the order-ing of the words and they also ignore semantics of the words. For example, “powerful, ” “strong” and “Paris ” are equally distant. In this paper, we propose Paragraph Vector, an unsupervised algo-rithm that learns fixed-length feature representa-tions from variable-length pieces of texts, such as sentences, paragraphs, and documents. Our algo-rithm represents each document by a dense vec-tor which is trained to predict words in the doc-ument. Its construction gives our algorithm the potential to overcome the weaknesses of bag-of-words models. Empirical results show that Para-graph Vectors outperform bag-of-words models as well as other techniques for text representa-tions. Finally, we achieve new state-of-the-art re-sults on several text classification and sentiment analysis tasks. 1.
Keyphrases
bag-of-words model bag-of-words feature para-graph vector sentiment analysis task unsupervised algo-rithm major weakness common fixed-length feature dense vec-tor empirical result fixed-length feature representa-tions variable-length piece text representa-tions many machine new state-of-the-art re-sults fixed-length feature vector paragraph vector several text classification
