Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings
This addresses a critical challenge for researchers and practitioners using LLMs in sequential tasks, though it is an incremental improvement over existing regularization-based approaches.
The paper tackles catastrophic forgetting in continual learning for large language models by revealing that drift of functional directions causes failures in existing methods, and proposes Dynamic Orthogonal Continual fine-tuning to track and adjust gradients, which outperforms prior methods in benchmarks.
Catastrophic forgetting remains a critical challenge in continual learning for large language models (LLMs), where models struggle to retain performance on historical tasks when fine-tuning on new sequential data without access to past datasets. In this paper, we first reveal that the drift of functional directions during the fine-tuning process is a key reason why existing regularization-based methods fail in long-term LLM continual learning. To address this, we propose Dynamic Orthogonal Continual (DOC) fine-tuning, a novel approach that tracks the drift of these functional directions and dynamically updates them during the fine-tuning process. Furthermore, by adjusting the gradients of new task parameters to be orthogonal to the tracked historical function directions, our method mitigates interference between new and old tasks. Extensive experiments on various LLM continual learning benchmarks demonstrate that this approach outperforms prior methods, effectively reducing catastrophic forgetting and providing a robust tool for continuous LLM fine-tuning. Our code is available at https://github.com/meloxxxxxx/DOC.