ReXTrust: A Model for Fine-Grained Hallucination Detection in AI-Generated Radiology Reports
This addresses the need for reliable hallucination detection to improve safety in automated radiology reporting, though it appears incremental as it builds on existing vision-language models.
The paper tackled the problem of detecting hallucinations in AI-generated radiology reports by introducing ReXTrust, a framework that achieved AUROC scores of 0.8751 across all findings and 0.8963 on clinically significant findings.
The increasing adoption of AI-generated radiology reports necessitates robust methods for detecting hallucinations--false or unfounded statements that could impact patient care. We present ReXTrust, a novel framework for fine-grained hallucination detection in AI-generated radiology reports. Our approach leverages sequences of hidden states from large vision-language models to produce finding-level hallucination risk scores. We evaluate ReXTrust on a subset of the MIMIC-CXR dataset and demonstrate superior performance compared to existing approaches, achieving an AUROC of 0.8751 across all findings and 0.8963 on clinically significant findings. Our results show that white-box approaches leveraging model hidden states can provide reliable hallucination detection for medical AI systems, potentially improving the safety and reliability of automated radiology reporting.