Learning to Reason with Third-Order Tensor Products
This addresses the challenge of systematic generalization in AI, particularly for natural language reasoning, though it appears incremental as it builds on existing methods like RNNs and Tensor Product Representations.
The paper tackled the problem of learning combinatorial representations for sequential data to improve symbolic interpretation and systematic generalization, achieving significant performance improvements over state-of-the-art models on natural language reasoning tasks.
We combine Recurrent Neural Networks with Tensor Product Representations to learn combinatorial representations of sequential data. This improves symbolic interpretation and systematic generalisation. Our architecture is trained end-to-end through gradient descent on a variety of simple natural language reasoning tasks, significantly outperforming the latest state-of-the-art models in single-task and all-tasks settings. We also augment a subset of the data such that training and test data exhibit large systematic differences and show that our approach generalises better than the previous state-of-the-art.