Revitalizing the Beginning: Avoiding Storage Dependency for Model Merging in Continual Learning
For continual learning researchers, this work addresses the storage and optimization challenges in model merging, offering a practical solution to improve multi-task model integration.
The paper identifies that existing model merging methods in continual learning suffer from error accumulation and vanishing gradients, leading to suboptimal merged models. The proposed Trajectory Regularized Merging (TRM) framework achieves state-of-the-art performance across multiple benchmarks.
Model merging provides a compelling paradigm for integrating specialized expertise into a unified multi-task model, a goal that aligns naturally with the sequential knowledge acquisition in continual learning (CL). However, the requirement for preserving diverse forms of previous knowledge conflicts with the storage limitations inherent to CL. In this paper, we systematically analyze existing model merging methods under the constraints of CL. We find that current methods prioritize global alignment, which often leads to the accumulation and amplification of task-specific errors within the continuous data stream; and the vanishing gradients at the onset of subsequent tasks frequently cause optimization to stagnate. These leave the merged model in a suboptimal state at the beginning of the next training phase. To address these challenges, we propose Trajectory Regularized Merging (TRM), a framework that reformulates the merging phase as an optimization process within an augmented trajectory subspace. Our framework integrates three synergistic objectives including task alignment, prediction consistency, and gradient responsiveness to concurrently preserve merged model's historical stability and re-activate optimization dynamics. Extensive experimental results demonstrate that our method achieves state-of-the-art performance across multiple benchmarks.