Influence Functions for Efficient Data Selection in Reasoning
This work addresses the challenge of efficiently selecting high-quality data for fine-tuning large language models on reasoning tasks, which is incremental as it applies an existing technique (influence functions) to a new context.
The paper tackled the problem of defining data quality for reasoning tasks by using influence functions to measure the causal effect of chain-of-thought examples on accuracy, resulting in influence-based pruning that consistently outperformed baselines like perplexity and embeddings on math reasoning.
Fine-tuning large language models (LLMs) on chain-of-thought (CoT) data shows that a small amount of high-quality data can outperform massive datasets. Yet, what constitutes "quality" remains ill-defined. Existing reasoning methods rely on indirect heuristics such as problem difficulty or trace length, while instruction-tuning has explored a broader range of automated selection strategies, but rarely in the context of reasoning. We propose to define reasoning data quality using influence functions, which measure the causal effect of individual CoT examples on downstream accuracy, and introduce influence-based pruning, which consistently outperforms perplexity and embedding-based baselines on math reasoning within a model family.