Empirical Evaluation of RNN Architectures on Sentence Classification Task
This work addresses the need for empirical evaluation of RNN architectures in NLP, but it is incremental as it builds on existing models without introducing a fundamentally new approach.
The paper tackled the problem of comparing RNN architectures for sentence classification by proposing a hybrid model and conducting an empirical study with LSTMs, finding that Max Pooling or Hybrid Max Pooling models performed best on most datasets, while the Tail Model did not outperform others.
Recurrent Neural Networks have achieved state-of-the-art results for many problems in NLP and two most popular RNN architectures are Tail Model and Pooling Model. In this paper, a hybrid architecture is proposed and we present the first empirical study using LSTMs to compare performance of the three RNN structures on sentence classification task. Experimental results show that the Max Pooling Model or Hybrid Max Pooling Model achieves the best performance on most datasets, while Tail Model does not outperform other models.