DPI: Exploiting Parameter Heterogeneity for Interference-Free Fine-Tuning
This addresses interference in multi-task fine-tuning for LLM adaptation, offering an incremental improvement to enhance model stability across heterogeneous tasks.
The paper tackled the seesaw effect in supervised fine-tuning of large language models, where conflicting tasks degrade performance, by proposing a dynamic parameter isolation strategy that identifies and freezes task-specific parameter regions, achieving consistent performance improvements over baselines.
Supervised fine-tuning (SFT) is a crucial step for adapting large language models (LLMs) to downstream tasks. However, conflicting objectives across heterogeneous SFT tasks often induce the "seesaw effect": optimizing for one task may degrade performance on others, particularly when model parameters are updated indiscriminately. In this paper, we propose a principled approach to disentangle and isolate task-specific parameter regions, motivated by the hypothesis that parameter heterogeneity underlies cross-task interference. Specifically, we first independently fine-tune LLMs on diverse SFT tasks and identify each task's core parameter region as the subset of parameters exhibiting the largest updates. Tasks with highly overlapping core parameter regions are merged for joint training, while disjoint tasks are organized into different stages. During multi-stage SFT, core parameters acquired in prior tasks are frozen, thereby preventing overwriting by subsequent tasks. To verify the effectiveness of our method, we conducted intensive experiments on multiple public datasets. The results showed that our dynamic parameter isolation strategy consistently reduced data conflicts and achieved consistent performance improvements compared to multi-stage and multi-task tuning baselines.