Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting
This addresses a problem for third-party practitioners fine-tuning open-source models, offering an incremental improvement by mitigating forgetting without original data.
The paper tackles catastrophic forgetting in supervised fine-tuning of large language models by proposing a method that reconstructs the base model's instruction distribution and synthesizes a general-purpose dataset, resulting in preserved general capabilities and improved task-specific performance compared to baselines.
Supervised Fine-Tuning (SFT) is a critical step for enhancing the instruction-following capabilities of Large Language Models (LLMs) and adapting them to specialized domains. However, SFT often leads to a degradation of the model's general abilities, a phenomenon known as catastrophic forgetting. This problem is exacerbated when third-party practitioners fine-tune open-source models, as the original SFT data is typically not available. To address this challenge, we propose a novel and cost-effective SFT method that effectively mitigates catastrophic forgetting without requiring access to the original SFT data. Our approach first reconstructs the likely instruction distribution of the base model. It then employs a multi-model generation and filtering pipeline to synthesize a high-quality general-purpose dataset. This synthetic dataset is mixed with new, domain-specific data for fine-tuning. Experimental results show that our method not only preserves the model's capabilities in general domains but also improves task-specific performance, outperforming baselines that use publicly available SFT datasets.