Tensor Decompositions in Recursive Neural Networks for Tree-Structured Data
This work addresses the challenge of efficiently processing tree-structured data in neural networks, though it appears incremental as it builds on existing tensor decomposition methods.
The paper tackled the problem of encoding structural knowledge from tree-structured data by introducing two new aggregation functions based on tensor decompositions, which limited model parameters while maintaining expressiveness. The resulting neural recursive models showed advantages on tree classification tasks, particularly when tree outdegree increased.
The paper introduces two new aggregation functions to encode structural knowledge from tree-structured data. They leverage the Canonical and Tensor-Train decompositions to yield expressive context aggregation while limiting the number of model parameters. Finally, we define two novel neural recursive models for trees leveraging such aggregation functions, and we test them on two tree classification tasks, showing the advantage of proposed models when tree outdegree increases.