When Chain-of-Thought Backfires: Evaluating Prompt Sensitivity in Medical Language Models
This work highlights critical prompt sensitivity issues for medical LLMs, showing that techniques validated on general models fail in this domain, which is important for developers and users in healthcare to ensure reliable deployment.
The study evaluated MedGemma models on medical QA datasets and found that Chain-of-Thought prompting decreased accuracy by 5.7%, while few-shot examples degraded performance by 11.9% and increased position bias, with cloze scoring achieving up to 64.5% accuracy, surpassing all prompting strategies.
Large Language Models (LLMs) are increasingly deployed in medical settings, yet their sensitivity to prompt formatting remains poorly characterized. We evaluate MedGemma (4B and 27B parameters) on MedMCQA (4,183 questions) and PubMedQA (1,000 questions) across a broad suite of robustness tests. Our experiments reveal several concerning findings. Chain-of-Thought (CoT) prompting decreases accuracy by 5.7% compared to direct answering. Few-shot examples degrade performance by 11.9% while increasing position bias from 0.14 to 0.47. Shuffling answer options causes the model to change predictions 59.1% of the time, with accuracy dropping up to 27.4 percentage points. Front-truncating context to 50% causes accuracy to plummet below the no-context baseline, yet back-truncation preserves 97% of full-context accuracy. We further show that cloze scoring (selecting the highest log-probability option token) achieves 51.8% (4B) and 64.5% (27B), surpassing all prompting strategies and revealing that models "know" more than their generated text shows. Permutation voting recovers 4 percentage points over single-ordering inference. These results demonstrate that prompt engineering techniques validated on general-purpose models do not transfer to domain-specific medical LLMs, and that reliable alternatives exist.