FroM: Frobenius Norm-Based Data-Free Adaptive Model Merging
This work addresses task interference in model merging for large language models, offering an incremental improvement over existing methods like RegMean.
The paper tackles the problem of task interference in merging multiple fine-tuned models by proposing FroM, a data-free adaptive method based on the Frobenius norm, which outperforms baseline methods in various fine-tuning scenarios.
With the development of large language models, fine-tuning has emerged as an effective method to enhance performance in specific scenarios by injecting domain-specific knowledge. In this context, model merging techniques provide a solution for fusing knowledge from multiple fine-tuning models by combining their parameters. However, traditional methods often encounter task interference when merging full fine-tuning models, and this problem becomes even more evident in parameter-efficient fine-tuning scenarios. In this paper, we introduce an improvement to the RegMean method, which indirectly leverages the training data to approximate the outputs of the linear layers before and after merging. We propose an adaptive merging method called FroM, which directly measures the model parameters using the Frobenius norm, without any training data. By introducing an additional hyperparameter for control, FroM outperforms baseline methods across various fine-tuning scenarios, alleviating the task interference problem.