Improving Tree-LSTM with Tree Attention
This work addresses the need for better tree-based models in NLP, but it is incremental as it builds on Tree-LSTM with attention improvements.
The paper tackled the problem of extracting information from tree structures in NLP by designing a generalized attention framework for Tree-LSTMs, achieving notable results on a semantic relatedness task compared to existing methods.
In Natural Language Processing (NLP), we often need to extract information from tree topology. Sentence structure can be represented via a dependency tree or a constituency tree structure. For this reason, a variant of LSTMs, named Tree-LSTM, was proposed to work on tree topology. In this paper, we design a generalized attention framework for both dependency and constituency trees by encoding variants of decomposable attention inside a Tree-LSTM cell. We evaluated our models on a semantic relatedness task and achieved notable results compared to Tree-LSTM based methods with no attention as well as other neural and non-neural methods and good results compared to Tree-LSTM based methods with attention.