MedReflect: Teaching Medical LLMs to Self-Improve via Reflective Correction
This addresses the need for cost-efficient and specialized medical AI by reducing reliance on external supervision and extensive data, though it is incremental as it builds on existing reflective thinking concepts.
The paper tackles the problem of medical problem-solving with large language models by introducing MedReflect, a framework that uses self-reflective correction to improve performance without external retrieval or heavy annotation, achieving notable absolute accuracy improvements across medical benchmarks with only 2,000 training examples.
Medical problem solving demands expert knowledge and intricate reasoning. Recent studies of large language models (LLMs) attempt to ease this complexity by introducing external knowledge verification through retrieval-augmented generation or by training on reasoning datasets. However, these approaches suffer from drawbacks such as retrieval overhead and high annotation costs, and they heavily rely on substituted external assistants to reach limited performance in medical field. In this paper, we introduce MedReflect, a generalizable framework designed to inspire LLMs with a physician-like reflective thinking mode. MedReflect generates a single-pass reflection chain that includes initial hypothesis generation, self-questioning, self-answering and decision refinement. This self-verified and self-reflective nature releases large language model's latent capability in medical problem-solving without external retrieval or heavy annotation. We demonstrate that MedReflect enables cost-efficient medical dataset construction: with merely 2,000 randomly sampled training examples and a light fine-tuning, this approach achieves notable absolute accuracy improvements across a series of medical benchmarks while cutting annotation requirements. Our results provide evidence that LLMs can learn to solve specialized medical problems via self-reflection and self-improve, reducing reliance on external supervision and extensive task-specific fine-tuning data.