Fact-Controlled Diagnosis of Hallucinations in Medical Text Summarization
This addresses the risk of hallucinations in patient-clinician dialogue summarization for clinical decision-making, offering specialized metrics and datasets, though it is incremental in improving detection methods.
The paper tackled the problem of hallucinations in large language models during medical text summarization by evaluating detection methods and constructing two datasets, showing that general-domain detectors struggle and that fact-based approaches offer explainability and generalize well to real-world clinical hallucinations.
Hallucinations in large language models (LLMs) during summarization of patient-clinician dialogues pose significant risks to patient care and clinical decision-making. However, the phenomenon remains understudied in the clinical domain, with uncertainty surrounding the applicability of general-domain hallucination detectors. The rarity and randomness of hallucinations further complicate their investigation. In this paper, we conduct an evaluation of hallucination detection methods in the medical domain, and construct two datasets for the purpose: A fact-controlled Leave-N-out dataset -- generated by systematically removing facts from source dialogues to induce hallucinated content in summaries; and a natural hallucination dataset -- arising organically during LLM-based medical summarization. We show that general-domain detectors struggle to detect clinical hallucinations, and that performance on fact-controlled hallucinations does not reliably predict effectiveness on natural hallucinations. We then develop fact-based approaches that count hallucinations, offering explainability not available with existing methods. Notably, our LLM-based detectors, which we developed using fact-controlled hallucinations, generalize well to detecting real-world clinical hallucinations. This research contributes a suite of specialized metrics supported by expert-annotated datasets to advance faithful clinical summarization systems.