DMCA
Global Belief Recursive Neural Networks
| Citations: | 4 - 0 self |
BibTeX
@MISC{Paulus_globalbelief,
author = {Romain Paulus and Richard Socher and Christopher D. Manning},
title = {Global Belief Recursive Neural Networks},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Recursive Neural Networks have recently obtained state of the art performance on several natural language processing tasks. However, because of their feedforward architecture they cannot correctly predict phrase or word labels that are deter-mined by context. This is a problem in tasks such as aspect-specific sentiment classification which tries to, for instance, predict that the word Android is positive in the sentence Android beats iOS. We introduce global belief recursive neural networks (GB-RNNs) which are based on the idea of extending purely feedfor-ward neural networks to include one feedbackward step during inference. This allows phrase level predictions and representations to give feedback to words. We show the effectiveness of this model on the task of contextual sentiment analy-sis. We also show that dropout can improve RNN training and that a combination of unsupervised and supervised word vector representations performs better than either alone. The feedbackward step improves F1 performance by 3 % over the standard RNN on this task, obtains state-of-the-art performance on the SemEval 2013 challenge and can accurately predict the sentiment of specific entities. 1
Keyphrases
global belief recursive neural network feedbackward step word label sentence android f1 performance several natural language processing task aspect-specific sentiment classification state-of-the-art performance feedfor-ward neural network feedforward architecture specific entity contextual sentiment analy-sis predict phrase word android supervised word vector representation art performance recursive neural network rnn training standard rnn phrase level prediction
