Efficient Data Selection at Scale via Influence Distillation
This addresses the problem of scalable and effective data selection for LLM fine-tuning, offering a novel method that is incremental in improving efficiency over existing approaches.
The paper tackles efficient data selection for fine-tuning large language models by introducing Influence Distillation, a framework that uses second-order information to weight training samples, achieving up to 3.5x faster selection while matching or outperforming state-of-the-art performance on tasks like GSM8k, SQuAD, and MMLU.
Effective data selection is critical for efficient training of modern Large Language Models (LLMs). This paper introduces Influence Distillation, a novel, mathematically-justified framework for data selection that employs second-order information to optimally weight training samples. By distilling each sample's influence on a target distribution, our method assigns model-specific weights that are used to select training data for LLM fine-tuning, guiding it toward strong performance on the target domain. We derive these optimal weights for both Gradient Descent and Adam optimizers. To ensure scalability and reduce computational cost, we propose a $\textit{landmark-based approximation}$: influence is precisely computed for a small subset of "landmark" samples and then efficiently propagated to all other samples to determine their weights. We validate Influence Distillation by applying it to instruction tuning on the Tulu V2 dataset, targeting a range of tasks including GSM8k, SQuAD, and MMLU, across several models from the Llama and Qwen families. Experiments show that Influence Distillation matches or outperforms state-of-the-art performance while achieving up to $3.5\times$ faster selection.