Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning
This addresses the cost and efficiency issue for researchers and practitioners fine-tuning large language models on long reasoning tasks, though it is incremental as it builds on existing instruction-tuning methods.
The paper tackles the problem of high training overhead from large-scale instruction datasets for long chain-of-thought reasoning by proposing Select2Reason, an efficient data selection framework; it shows that fine-tuning on only 10% of selected data achieves performance competitive or superior to full-data tuning across multiple benchmarks.
A practical approach to activate long chain-of-thoughts reasoning ability in pre-trained large language models is to perform supervised fine-tuning on instruction datasets synthesized by strong Large Reasoning Models such as DeepSeek-R1, offering a cost-effective alternative to reinforcement learning. However, large-scale instruction sets with more than 100k samples incur significant training overhead, while effective strategies for automatic long-CoT instruction selection still remain unexplored. In this work, we propose Select2Reason, a novel and efficient instruction-tuning data selection framework for long-CoT reasoning. From the perspective of emergence of rethinking behaviors like self-correction and backtracking, we investigate common metrics that may determine the quality of long-CoT reasoning instructions. Select2Reason leverages a quantifier to estimate difficulty of question and jointly incorporates a reasoning trace length-based heuristic through a weighted scheme for ranking to prioritize high-utility examples. Empirical results on OpenR1-Math-220k demonstrate that fine-tuning LLM on only 10% of the data selected by Select2Reason achieves performance competitive with or superior to full-data tuning and open-source baseline OpenR1-Qwen-7B across three competition-level and six comprehensive mathematical benchmarks. Further experiments highlight the scalability in varying data size, efficiency during inference, and its adaptability to other instruction pools with minimal cost.