Learning to Embed Sentences Using Attentive Recursive Trees
This work addresses sentence embedding for NLP tasks by introducing a mechanism to highlight important words, though it appears incremental as it builds on existing recursive tree methods.
The paper tackled the problem of sentence embedding by proposing an Attentive Recursive Tree model (AR-Tree) that dynamically emphasizes task-informative words in tree structures, resulting in consistent outperformance or comparability to state-of-the-art methods on three sentence understanding tasks.
Sentence embedding is an effective feature representation for most deep learning-based NLP tasks. One prevailing line of methods is using recursive latent tree-structured networks to embed sentences with task-specific structures. However, existing models have no explicit mechanism to emphasize task-informative words in the tree structure. To this end, we propose an Attentive Recursive Tree model (AR-Tree), where the words are dynamically located according to their importance in the task. Specifically, we construct the latent tree for a sentence in a proposed important-first strategy, and place more attentive words nearer to the root; thus, AR-Tree can inherently emphasize important words during the bottom-up composition of the sentence embedding. We propose an end-to-end reinforced training strategy for AR-Tree, which is demonstrated to consistently outperform, or be at least comparable to, the state-of-the-art sentence embedding methods on three sentence understanding tasks.