Improving Compositional Generalization in Classification Tasks via Structure Annotations
This addresses a key limitation in neural models for systematic generalization, but the approach appears incremental as it builds on existing Transformer methods with added annotations.
The paper tackles the problem of compositional generalization in classification tasks by converting sequence-to-sequence datasets to classification formats and demonstrating that structural hints like parse trees and entity links as attention masks improve performance, though no concrete numbers are provided.
Compositional generalization is the ability to generalize systematically to a new data distribution by combining known components. Although humans seem to have a great ability to generalize compositionally, state-of-the-art neural models struggle to do so. In this work, we study compositional generalization in classification tasks and present two main contributions. First, we study ways to convert a natural language sequence-to-sequence dataset to a classification dataset that also requires compositional generalization. Second, we show that providing structural hints (specifically, providing parse trees and entity links as attention masks for a Transformer model) helps compositional generalization.