Learning Document Embeddings by Predicting N-grams for Sentiment Classification of Long Movie Reviews
This addresses sentiment classification for movie reviews, offering an incremental improvement by combining semantics with n-gram features.
The paper tackled the problem of sentiment classification for long movie reviews by modifying Paragraph Vector to predict n-gram features, capturing both semantics and word order, and achieved outperformance over previous deep learning and bag-of-ngram models on the IMDB dataset.
Despite the loss of semantic information, bag-of-ngram based methods still achieve state-of-the-art results for tasks such as sentiment classification of long movie reviews. Many document embeddings methods have been proposed to capture semantics, but they still can't outperform bag-of-ngram based methods on this task. In this paper, we modify the architecture of the recently proposed Paragraph Vector, allowing it to learn document vectors by predicting not only words, but n-gram features as well. Our model is able to capture both semantics and word order in documents while keeping the expressive power of learned vectors. Experimental results on IMDB movie review dataset shows that our model outperforms previous deep learning models and bag-of-ngram based models due to the above advantages. More robust results are also obtained when our model is combined with other models. The source code of our model will be also published together with this paper.