Not All Parameters Are Created Equal: Smart Isolation Boosts Fine-Tuning Performance
This addresses performance degradation in multi-task fine-tuning for LLM adaptation, offering a method to mitigate task interference and forgetting, though it is incremental as it builds on existing fine-tuning techniques.
The paper tackles the seesaw phenomenon in supervised fine-tuning of large language models, where indiscriminate parameter updates cause performance trade-offs across tasks, and proposes a Core Parameter Isolation Fine-Tuning framework that groups tasks by core parameter regions and fuses parameters to reduce interference, achieving significant improvements over baselines on multiple benchmarks.
Supervised fine-tuning (SFT) is a pivotal approach to adapting large language models (LLMs) for downstream tasks; however, performance often suffers from the ``seesaw phenomenon'', where indiscriminate parameter updates yield progress on certain tasks at the expense of others. To address this challenge, we propose a novel \emph{Core Parameter Isolation Fine-Tuning} (CPI-FT) framework. Specifically, we first independently fine-tune the LLM on each task to identify its core parameter regions by quantifying parameter update magnitudes. Tasks with similar core regions are then grouped based on region overlap, forming clusters for joint modeling. We further introduce a parameter fusion technique: for each task, core parameters from its individually fine-tuned model are directly transplanted into a unified backbone, while non-core parameters from different tasks are smoothly integrated via Spherical Linear Interpolation (SLERP), mitigating destructive interference. A lightweight, pipelined SFT training phase using mixed-task data is subsequently employed, while freezing core regions from prior tasks to prevent catastrophic forgetting. Extensive experiments on multiple public benchmarks demonstrate that our approach significantly alleviates task interference and forgetting, consistently outperforming vanilla multi-task and multi-stage fine-tuning baselines.