Compositional Generalization Requires Compositional Parsers
This addresses the challenge of enabling AI systems to handle novel syntactic structures, which is crucial for robust natural language understanding, though the findings are incremental as they confirm prior results from compositional parsers.
The study tackled the problem of compositional generalization in semantic parsing by comparing sequence-to-sequence models with compositional models on the COGS corpus, finding that seq2seq models had near-zero accuracy on structural generalization tasks while compositional models achieved near-perfect accuracy.
A rapidly growing body of research on compositional generalization investigates the ability of a semantic parser to dynamically recombine linguistic elements seen in training into unseen sequences. We present a systematic comparison of sequence-to-sequence models and models guided by compositional principles on the recent COGS corpus (Kim and Linzen, 2020). Though seq2seq models can perform well on lexical tasks, they perform with near-zero accuracy on structural generalization tasks that require novel syntactic structures; this holds true even when they are trained to predict syntax instead of semantics. In contrast, compositional models achieve near-perfect accuracy on structural generalization; we present new results confirming this from the AM parser (Groschwitz et al., 2021). Our findings show structural generalization is a key measure of compositional generalization and requires models that are aware of complex structure.