Modeling Compositionality with Multiplicative Recurrent Neural Networks
This work addresses the challenge of compositional meaning in language for natural language processing tasks, offering a novel approach that is incremental in improving existing methods.
The authors tackled the problem of modeling compositional meaning in language by proposing multiplicative recurrent neural networks, achieving comparable or better performance than additive recurrent networks and matrix-space models on fine-grained sentiment analysis, with results matching structural deep models on the Stanford Sentiment Treebank without requiring parse trees.
We present the multiplicative recurrent neural network as a general model for compositional meaning in language, and evaluate it on the task of fine-grained sentiment analysis. We establish a connection to the previously investigated matrix-space models for compositionality, and show they are special cases of the multiplicative recurrent net. Our experiments show that these models perform comparably or better than Elman-type additive recurrent neural networks and outperform matrix-space models on a standard fine-grained sentiment analysis corpus. Furthermore, they yield comparable results to structural deep models on the recently published Stanford Sentiment Treebank without the need for generating parse trees.