Rethinking Radiology Report Generation via Causal Inspired Counterfactual Augmentation
This work addresses a domain-specific issue in biomedical imaging by improving report accuracy for radiology applications, though it is incremental as it builds on existing RRG methods with a novel causal approach.
The paper tackled the problem of Radiology Report Generation (RRG) models being confused by disease co-occurrence due to biased data, by modeling causal effects and proposing a counterfactual augmentation method to intervene on backdoor paths, resulting in enhanced accuracy and generalization as demonstrated on the MIMIC-CXR and IU X-Ray datasets.
Radiology Report Generation (RRG) draws attention as a vision-and-language interaction of biomedical fields. Previous works inherited the ideology of traditional language generation tasks, aiming to generate paragraphs with high readability as reports. Despite significant progress, the independence between diseases-a specific property of RRG-was neglected, yielding the models being confused by the co-occurrence of diseases brought on by the biased data distribution, thus generating inaccurate reports. In this paper, to rethink this issue, we first model the causal effects between the variables from a causal perspective, through which we prove that the co-occurrence relationships between diseases on the biased distribution function as confounders, confusing the accuracy through two backdoor paths, i.e. the Joint Vision Coupling and the Conditional Sequential Coupling. Then, we proposed a novel model-agnostic counterfactual augmentation method that contains two strategies, i.e. the Prototype-based Counterfactual Sample Synthesis (P-CSS) and the Magic-Cube-like Counterfactual Report Reconstruction (Cube), to intervene the backdoor paths, thus enhancing the accuracy and generalization of RRG models. Experimental results on the widely used MIMIC-CXR dataset demonstrate the effectiveness of our proposed method. Additionally, a generalization performance is evaluated on IU X-Ray dataset, which verifies our work can effectively reduce the impact of co-occurrences caused by different distributions on the results.