PAFT: Prompt-Agnostic Fine-Tuning
This addresses a practical problem for users of fine-tuned LLMs who need consistent performance across prompt variations, representing a strong incremental improvement in fine-tuning methodology.
The paper tackles the problem of large language models overfitting to specific prompt wording during fine-tuning, proposing Prompt-Agnostic Fine-Tuning (PAFT) which improves robustness by using dynamic prompt variation during training. Results include 7% higher generalization accuracy on unseen prompts and 3.2× faster inference speeds compared to standard methods.
Fine-tuning large language models (LLMs) often causes overfitting to specific prompt wording, where minor phrasing variations drastically reduce performance. To address this, we propose Prompt-Agnostic Fine-Tuning (PAFT), a method that enhances robustness through dynamic prompt variation during training. PAFT first generates diverse synthetic prompts, then continuously samples from this set to construct training instances, forcing models to learn fundamental task principles rather than surface-level patterns. Across systematic evaluations using both supervised fine-tuning (SFT) and reinforcement learning fine-tuning (RLFT), PAFT demonstrates substantially improved prompt robustness, achieving 7% higher generalization accuracy on unseen prompts than standard methods. In addition to enhanced robustness, PAFT consistently yields superior overall performance on established benchmarks for question answering, mathematical reasoning, and tool use. Notably, models trained with PAFT attain 3.2 faster inference speeds due to reduced prompt sensitivity. Ablation studies further validate effectiveness of PAFT, while theoretical analysis reveals that PAFT can effectively enhance the cross-domain generalization ability of LLM.