Structural generalization in COGS: Supertagging is (almost) all you need
This addresses a key limitation in semantic parsing for NLP applications, though it is incremental as it builds on existing frameworks.
The paper tackles the problem of neural networks failing to generalize on out-of-distribution examples requiring compositional generalization in semantic parsing, by extending a neural graph-based framework with supertagging and other techniques, resulting in significant improvements on the COGS dataset.
In many Natural Language Processing applications, neural networks have been found to fail to generalize on out-of-distribution examples. In particular, several recent semantic parsing datasets have put forward important limitations of neural networks in cases where compositional generalization is required. In this work, we extend a neural graph-based semantic parsing framework in several ways to alleviate this issue. Notably, we propose: (1) the introduction of a supertagging step with valency constraints, expressed as an integer linear program; (2) a reduction of the graph prediction problem to the maximum matching problem; (3) the design of an incremental early-stopping training strategy to prevent overfitting. Experimentally, our approach significantly improves results on examples that require structural generalization in the COGS dataset, a known challenging benchmark for compositional generalization. Overall, our results confirm that structural constraints are important for generalization in semantic parsing.