Phrase-grounded Fact-checking for Automatically Generated Chest X-ray Reports
This addresses the issue of unreliable AI-generated radiology reports for clinical use, though it is incremental as it builds on existing vision-language models and error detection methods.
The paper tackles the problem of factual errors and hallucinations in automatically generated chest X-ray reports by developing a phrase-grounded fact-checking model, which achieves a concordance correlation coefficient of 0.997 with ground truth-based verification for error detection in reports from state-of-the-art generators.
With the emergence of large-scale vision language models (VLM), it is now possible to produce realistic-looking radiology reports for chest X-ray images. However, their clinical translation has been hampered by the factual errors and hallucinations in the produced descriptions during inference. In this paper, we present a novel phrase-grounded fact-checking model (FC model) that detects errors in findings and their indicated locations in automatically generated chest radiology reports. Specifically, we simulate the errors in reports through a large synthetic dataset derived by perturbing findings and their locations in ground truth reports to form real and fake findings-location pairs with images. A new multi-label cross-modal contrastive regression network is then trained on this dataset. We present results demonstrating the robustness of our method in terms of accuracy of finding veracity prediction and localization on multiple X-ray datasets. We also show its effectiveness for error detection in reports of SOTA report generators on multiple datasets achieving a concordance correlation coefficient of 0.997 with ground truth-based verification, thus pointing to its utility during clinical inference in radiology workflows.