Modelling Sentence Pairs with Tree-structured Attentive Encoder
This work addresses the challenge of improving natural language processing tasks like semantic similarity and paraphrase identification for researchers and practitioners, though it is incremental as it builds on existing attentive models by adding tree-structured attention.
The paper tackled the problem of modeling sentence pairs by proposing an attentive encoder that integrates tree-structured recursive neural networks with sequential RNNs, incorporating attention into tree topologies rather than just sequential structures. It achieved state-of-the-art results on semantic similarity and paraphrase identification tasks, outperforming all baselines.
We describe an attentive encoder that combines tree-structured recursive neural networks and sequential recurrent neural networks for modelling sentence pairs. Since existing attentive models exert attention on the sequential structure, we propose a way to incorporate attention into the tree topology. Specially, given a pair of sentences, our attentive encoder uses the representation of one sentence, which generated via an RNN, to guide the structural encoding of the other sentence on the dependency parse tree. We evaluate the proposed attentive encoder on three tasks: semantic similarity, paraphrase identification and true-false question selection. Experimental results show that our encoder outperforms all baselines and achieves state-of-the-art results on two tasks.