AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learning
This addresses the problem of efficient continual learning for large language models in dynamic environments, representing a novel method for a known bottleneck.
The paper tackles the challenge of balancing learning new knowledge and preventing forgetting in continual learning for large language models by introducing AimMerging, which uses training trajectories to adaptively control model merging, resulting in average relative improvements of 80% and 59% on FWT and BWT metrics.
Continual learning (CL) is essential for deploying large language models (LLMs) in dynamic real-world environments without the need for costly retraining. Recent model merging-based methods have attracted significant attention, but they still struggle to effectively manage the trade-off between learning new knowledge and preventing forgetting, a challenge largely stemming from suboptimal number of merges and merging frequency. In this paper, we introduce Adaptive Iterative Model Merging (AimMerging), a novel CL framework that utilizes learning and forgetting signals from the training trajectory to dynamically monitor the model's training status. Guided by dynamic monitoring, the training trajectory-guided merge controller adaptively determines the timing and frequency of iterative fusion, while the rehearsal-based knowledge fusion module computes the merging weights and executes the fusion. Comprehensive experiments on three CL benchmarks with various model sizes (from 770M to 13B) demonstrate that AimMerging achieves significant performance improvements over existing state-of-the-art methods, with an average relative improvement of 80% and 59% on FWT and BWT, respectively. The source code is provided for reproducibility.