Student-in-the-Loop Chain-of-Thought Distillation via Generation-Time Selection
This addresses the problem of inefficient knowledge distillation for reasoning tasks in machine learning, though it is incremental as it builds on existing distillation methods.
The paper tackled the challenge of transferring reasoning processes from large to small models by proposing a student-in-the-loop framework that selects learnable reasoning paths during generation, resulting in improvements of around 5.9 points over standard knowledge distillation and up to 4.7 points over other baselines on mathematical reasoning benchmarks.
Large reasoning models achieve strong performance on complex tasks through long chain-of-thought (CoT) trajectories, but directly transferring such reasoning processes to smaller models remains challenging. A key difficulty is that not all teacher-generated reasoning trajectories are suitable for student learning. Existing approaches typically rely on post-hoc filtering, selecting trajectories after full generation based on heuristic criteria. However, such methods cannot control the generation process itself and may still produce reasoning paths that lie outside the student's learning capacity. To address this limitation, we propose Gen-SSD (Generation-time Self-Selection Distillation), a student-in-the-loop framework that performs generation-time selection. Instead of passively consuming complete trajectories, the student evaluates candidate continuations during the teacher's sampling process, guiding the expansion of only learnable reasoning paths and enabling early pruning of unhelpful branches. Experiments on mathematical reasoning benchmarks demonstrate that Gen-SSD consistently outperforms standard knowledge distillation and recent baselines, with improvements of around 5.9 points over Standard KD and up to 4.7 points over other baselines. Further analysis shows that Gen-SSD produces more stable and learnable reasoning trajectories, highlighting the importance of incorporating supervision during generation for effective distillation.