Unified Data Selection for LLM Reasoning
For researchers and practitioners training LLMs on reasoning tasks, HES provides a computationally efficient and effective method for data selection, addressing a key bottleneck in developing advanced reasoning capabilities.
The paper proposes High-Entropy Sum (HES), a training-free metric that selects high-quality reasoning data for LLMs by summing the entropy of the top highest-entropy tokens. HES consistently improves performance across SFT, RFT, and RL paradigms while reducing computational overhead, e.g., matching full-dataset SFT performance with only the top 20% of data.
Effectively training Large Language Models (LLMs) for complex, long-CoT reasoning is often bottlenecked by the need for massive high-quality reasoning data. Existing methods are either computationally expensive or fail to reliably distinguish high- from low-quality reasoning samples. To address this, we propose High-Entropy Sum (HES), a training-free metric that quantifies reasoning quality by summing only the entropy of the top (e.g., 0.5\%) highest-entropy tokens in each reasoning sample. We validate HES across three mainstream training paradigms: Supervised Fine-tuning (SFT), Rejection Fine-tuning (RFT), and Reinforcement Learning (RL), with extensive results demonstrating its consistent effectiveness and significantly reduced computational overhead. In SFT, training on the top 20\% HES-ranked data matches full-dataset performance, while using the lowest-HES data degrades it. In RFT, our HES-based training approach significantly outperforms baseline methods. In RL, HES-selected successful trajectories enable the model to learn strong reasoning patterns, significantly surpassing other compared methods. Our findings establish HES as a robust, training-free metric that enables a unified, effective, and efficient method for developing advanced reasoning in LLMs.