Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs
This work addresses the need for unsupervised syntax-aware sentence representations in NLP, offering a novel integration that eliminates dependency on external parse trees.
The paper tackles the problem of learning sentence embeddings without requiring external parse trees by jointly optimizing a Tree-LSTM composition function and a differentiable parser in an unsupervised manner, achieving better performance than supervised Tree-LSTM architectures on textual entailment and reverse dictionary tasks.
We introduce a neural network that represents sentences by composing their words according to induced binary parse trees. We use Tree-LSTM as our composition function, applied along a tree structure found by a fully differentiable natural language chart parser. Our model simultaneously optimises both the composition function and the parser, thus eliminating the need for externally-provided parse trees which are normally required for Tree-LSTM. It can therefore be seen as a tree-based RNN that is unsupervised with respect to the parse trees. As it is fully differentiable, our model is easily trained with an off-the-shelf gradient descent method and backpropagation. We demonstrate that it achieves better performance compared to various supervised Tree-LSTM architectures on a textual entailment task and a reverse dictionary task.