Exploring Continual Learning of Compositional Generalization in NLI
This addresses the challenge of making NLI models more human-like by learning incrementally, though it is incremental as it builds on existing continual learning methods for a specific domain.
The paper tackles the problem of compositional generalization in natural language inference (NLI) under continual learning, where models acquire primitive inference knowledge sequentially rather than all at once. The results show that models initially fail in this scenario, but by ordering subtasks based on dependencies and difficulty, continual learning can enhance compositional generalization.
Compositional Natural Language Inference has been explored to assess the true abilities of neural models to perform NLI. Yet, current evaluations assume models to have full access to all primitive inferences in advance, in contrast to humans that continuously acquire inference knowledge. In this paper, we introduce the Continual Compositional Generalization in Inference (C2Gen NLI) challenge, where a model continuously acquires knowledge of constituting primitive inference tasks as a basis for compositional inferences. We explore how continual learning affects compositional generalization in NLI, by designing a continual learning setup for compositional NLI inference tasks. Our experiments demonstrate that models fail to compositionally generalize in a continual scenario. To address this problem, we first benchmark various continual learning algorithms and verify their efficacy. We then further analyze C2Gen, focusing on how to order primitives and compositional inference types and examining correlations between subtasks. Our analyses show that by learning subtasks continuously while observing their dependencies and increasing degrees of difficulty, continual learning can enhance composition generalization ability.