Finetune Once: Decoupling General & Domain Learning with Dynamic Boosted Annealing
This addresses the problem of costly and complex fine-tuning for LLM practitioners by offering an efficient method, though it appears incremental as it builds on existing fine-tuning approaches.
The paper tackled the inefficiency of vanilla fine-tuning for large language models by proposing Dynamic Boosted Annealing (DBA), which decouples general and domain learning to streamline training, resulting in an average 5.8% improvement in joint performance and a 91.0% reduction in GPU hours.
Large language models (LLMs) fine-tuning shows excellent implications. However, vanilla fine-tuning methods often require intricate data mixture and repeated experiments for optimal generalization. To address these challenges and streamline the training process, we propose an efficient and universal solution, Dynamic Boosted Annealing (DBA). We obtain a global gradient through zero-learning-rate training on general data, which is subsequently employed for gradient boosting and dynamic training step correction during domain training. In conjunction with annealing learning, we end up establishing a fine-tuning pipeline that relies solely on domain data without collapse. By evaluating both general and domain-specific performance across multiple tasks on several popular base models, DBA achieves an average improvement of 5.8% in joint performance over vanilla fine-tuning. Furthermore, since general data is no longer involved in annealing, repeated experiments led by data mixture are also eliminated. According to our tests, the DBA method can reduce GPU hours by 91.0% compared to the vanilla method.