Compositional Distributional Semantics with Long Short Term Memory
This work addresses a specific bottleneck in natural language processing for sentiment analysis, though it is incremental as it builds on existing neural network architectures.
The authors tackled the problem of capturing long-range dependencies in compositional distributional semantics by extending recursive neural networks with a long short-term memory variant, resulting in improved performance on the Stanford Sentiment Treebank compared to traditional methods.
We are proposing an extension of the recursive neural network that makes use of a variant of the long short-term memory architecture. The extension allows information low in parse trees to be stored in a memory register (the `memory cell') and used much later higher up in the parse tree. This provides a solution to the vanishing gradient problem and allows the network to capture long range dependencies. Experimental results show that our composition outperformed the traditional neural-network composition on the Stanford Sentiment Treebank.