Spurious Forgetting in Continual Learning of Language Models
This addresses the issue of spurious forgetting in continual learning for language models, which is incremental as it builds on existing understanding of task alignment.
The study tackled the problem of performance declines in continual learning for language models, showing that these drops often result from disrupted task alignment rather than knowledge loss, and introduced a Freezing strategy that improved performance in four scenarios.
Recent advancements in large language models (LLMs) reveal a perplexing phenomenon in continual learning: despite extensive training, models experience significant performance declines, raising questions about task alignment and underlying knowledge retention. This study first explores the concept of "spurious forgetting", proposing that such performance drops often reflect a decline in task alignment rather than true knowledge loss. Through controlled experiments with a synthesized dataset, we investigate the dynamics of model performance during the initial training phases of new tasks, discovering that early optimization steps can disrupt previously established task alignments. Our theoretical analysis connects these shifts to orthogonal updates in model weights, providing a robust framework for understanding this behavior. Ultimately, we introduce a Freezing strategy that fix the bottom layers of the model, leading to substantial improvements in four continual learning scenarios. Our findings underscore the critical distinction between task alignment and knowledge retention, paving the way for more effective strategies in continual learning.