Beyond Literal Summarization: Redefining Hallucination for Medical SOAP Note Evaluation
For researchers and practitioners evaluating LLMs in clinical documentation, this work highlights the need for clinically informed evaluation to avoid penalizing valid reasoning and measuring artifacts instead of true errors.
Current evaluation methods for LLM-generated SOAP notes misclassify clinically valid outputs as hallucinations due to reliance on lexical faithfulness, inflating hallucination rates. By aligning evaluation with clinical reasoning, the hallucination rate drops from 35% to 9%, revealing that most flagged errors are actually legitimate clinical transformations.
Evaluating large language models (LLMs) for clinical documentation tasks such as SOAP note generation remains challenging. Unlike standard summarization, these tasks require clinical abstraction, normalization of colloquial language, and medically grounded inference. However, prevailing evaluation methods including automated metrics and LLM as judge frameworks rely on lexical faithfulness, often labeling any information not explicitly present in the transcript as hallucination. We show that such approaches systematically misclassify clinically valid outputs as errors, inflating hallucination rates and distorting model assessment. Our analysis reveals that many flagged hallucinations correspond to legitimate clinical transformations, including synonym mapping, abstraction of examination findings, diagnostic inference, and guideline consistent care planning. By aligning evaluation criteria with clinical reasoning through calibrated prompting and retrieval grounded in medical ontologies we observe a significant shift in outcomes. Under a lexical evaluation regime, the mean hallucination rate is 35%, heavily penalizing valid reasoning. With inference aware evaluation, this drops to 9%, with remaining cases reflecting genuine safety concerns. These findings suggest that current evaluation practices over penalize valid clinical reasoning and may measure artifacts of evaluation design rather than true errors, underscoring the need for clinically informed evaluation in high context domains like medicine.