Learning from Non-Binary Constituency Trees via Tensor Decomposition
This work addresses a limitation in NLP for researchers and practitioners by enabling direct use of non-binary trees without binarization, though it is incremental as it builds on existing tensor and Tree-LSTM methods.
The paper tackles the problem of processing non-binary constituency trees in NLP by introducing a tensor decomposition-based composition function, resulting in a Tree-LSTM model that shows improved performance on tasks like sentiment analysis and natural language inference, with gains such as a 1.2% accuracy increase on the SST dataset.
Processing sentence constituency trees in binarised form is a common and popular approach in literature. However, constituency trees are non-binary by nature. The binarisation procedure changes deeply the structure, furthering constituents that instead are close. In this work, we introduce a new approach to deal with non-binary constituency trees which leverages tensor-based models. In particular, we show how a powerful composition function based on the canonical tensor decomposition can exploit such a rich structure. A key point of our approach is the weight sharing constraint imposed on the factor matrices, which allows limiting the number of model parameters. Finally, we introduce a Tree-LSTM model which takes advantage of this composition function and we experimentally assess its performance on different NLP tasks.