MedCritical: Enhancing Medical Reasoning in Small Language Models via Self-Collaborative Correction
This work addresses the challenge of improving medical reasoning for small language models, offering a more cost-effective alternative to teacher-guided methods, though it is incremental as it builds on existing knowledge distillation and DPO techniques.
The paper tackles the problem of enhancing medical reasoning in small language models, which often underperform on complex tasks like clinical diagnosis, by proposing a self-collaborative correction framework that achieves new state-of-the-art performance, with the MedCritical 7B model outperforming Taiyi and Huatuo-o1-7B by 3.04% and 10.12% on the CMExam benchmark.
In the field of medicine, complex reasoning tasks such as clinical diagnosis, treatment planning, and medical knowledge integration pose significant challenges, where small language models often underperform compared to large language models like GPT-4 and Deepseek. Recent knowledge distillation-based methods aim to address these issues through teacher-guided error correction, but this LLM as judge approach remains challenging in terms of cost, time, and efficiency. To circumvent this issue, we propose a novel two-stage framework, MedCritical, which uses a small language model fine-tuned by a large teacher model to play against itself. In the first stage, we extract high-level and detailed long-chain thought templates from the teacher model to guide the student model to generate more complex reasoning thoughts. In the second stage, we introduce direct preference optimization (DPO) through model self-iteration collaboration to enhance the reasoning ability of the student model by playing against the correction trajectory of the fine-tuned model during training. This model self-learning DPO approach teaches the student model to use its own error-driven insights to consolidate its skills and knowledge to solve complex problems, and achieves comparable results to traditional knowledge distillation methods using teacher models at a lower cost. Notably, our MedCritical 7B model outperforms the Taiyi and Huatuo-o1-7B models by 3.04\% and 10.12\% respectively on the CMExam benchmark, achieving new SOTA performance among 7B-class small models.