Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation
This addresses the problem of inefficient adaptation of draft models for domain-specific fine-tuned LLMs, offering a parameter- and data-efficient solution.
The paper tackles the performance degradation of speculative decoding when target models are fine-tuned for specific domains by introducing Efficient Draft Adaptation (EDA), which restores speculative performance with superior average acceptance lengths and significantly reduced training costs compared to full retraining.
Speculative decoding accelerates LLM inference but suffers from performance degradation when target models are fine-tuned for specific domains. A naive solution is to retrain draft models for every target model, which is costly and inefficient. To address this, we introduce a parameter- and data-efficient framework named Efficient Draft Adaptation, abbreviated as EDA, for efficiently adapting draft models. EDA introduces three innovations: (1) a decoupled architecture that utilizes shared and private components to model the shared and target-specific output distributions separately, enabling parameter-efficient adaptation by updating only the lightweight private component;(2) a data regeneration strategy that utilizes the fine-tuned target model to regenerate training data, thereby improving the alignment between training and speculative decoding, leading to higher average acceptance length;(3) a sample selection mechanism that prioritizes high-value data for efficient adaptation. Our experiments show that EDA effectively restores speculative performance on fine-tuned models, achieving superior average acceptance lengths with significantly reduced training costs compared to full retraining. Code is available at https://github.com/Lyn-Lucy/Efficient-Draft-Adaptation.