Position Paper: Generalized grammar rules and structure-based generalization beyond classical equivariance for lexical tasks and transduction
This addresses a key limitation in AI for language processing, though it appears incremental as it builds on existing symmetry concepts.
The paper tackles the problem of compositional generalization in neural networks for lexical tasks by proposing Generalized Grammar Rules (GGRs), a symmetry-based framework for transduction, which unifies existing works and suggests implementation ideas.
Compositional generalization is one of the main properties which differentiates lexical learning in humans from state-of-art neural networks. We propose a general framework for building models that can generalize compositionally using the concept of Generalized Grammar Rules (GGRs), a class of symmetry-based compositional constraints for transduction tasks, which we view as a transduction analogue of equivariance constraints in physics-inspired tasks. Besides formalizing generalized notions of symmetry for language transduction, our framework is general enough to contain many existing works as special cases. We present ideas on how GGRs might be implemented, and in the process draw connections to reinforcement learning and other areas of research.