CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning
This addresses the challenge of generating reliable scientific reasoning data for training smaller models, though it is incremental as it builds on existing CoT distillation methods.
The paper tackled the problem of low-quality chain-of-thought distillation for scientific reasoning by proposing CoT-Evo, an evolutionary framework that refines reasoning trajectories to create a high-quality dataset, resulting in a compact model achieving state-of-the-art performance on scientific reasoning benchmarks.
While chain-of-thought (CoT) distillation from advanced large language models (LLMs) has proven effective in general reasoning tasks, it struggles in scientific domains where even advanced models often produce incorrect or superficial reasoning due to high complexity and specialized knowledge requirements. Directly distilling from such flawed outputs results in low-quality training data and limits the performance of smaller student models. To overcome this, we propose CoT-Evo, an evolutionary CoT distillation framework. It begins by constructing a diverse pool of reasoning trajectories from multiple LLM thinkers, enriches them with automatically retrieved domain knowledge, and iteratively refines the trajectories using novelty-driven selection, reflective recombination and mutation. The refinement is guided by a fitness function that evaluates answer correctness, coherence, and effective knowledge utilization. This results in a high-quality CoT dataset tailored for scientific reasoning. We employ this evolved dataset to fine-tune a compact model, which achieves state-of-the-art performance on scientific reasoning benchmarks. Our work establishes a scalable approach to synthesizing high-fidelity scientific reasoning data from diverse and fallible LLMs.