Bidirectional Tree-Structured LSTM with Head Lexicalization
This work addresses a limitation in NLP models for tasks like sentiment analysis and question classification, though it is incremental as it builds on existing tree LSTM methods.
The paper tackled the problem of tree-structured LSTMs lacking direct input word references for non-leaf nodes by proposing automatic head lexicalization and a top-down tree LSTM, resulting in improved tree representations and achieving state-of-the-art results on the Stanford Sentiment Treebank and competitive results on TREC question classification.
Sequential LSTM has been extended to model tree structures, giving competitive results for a number of tasks. Existing methods model constituent trees by bottom-up combinations of constituent nodes, making direct use of input word information only for leaf nodes. This is different from sequential LSTMs, which contain reference to input words for each node. In this paper, we propose a method for automatic head-lexicalization for tree-structure LSTMs, propagating head words from leaf nodes to every constituent node. In addition, enabled by head lexicalization, we build a tree LSTM in the top-down direction, which corresponds to bidirectional sequential LSTM structurally. Experiments show that both extensions give better representations of tree structures. Our final model gives the best results on the Standford Sentiment Treebank and highly competitive results on the TREC question type classification task.