The Effect of Counterfactuals on Reading Chest X-rays
This addresses the problem of improving interpretability and trust in AI predictions for radiologists, but it is incremental as it builds on existing explanation methods.
The study investigated how counterfactual explanations affect radiologists' confidence in chest X-ray predictions, finding that they increased confidence in true positives by 0.15±0.95 (p=0.01) with a minor increase in false positives of 0.04±1.06 (p=0.57), particularly benefiting tasks like Mass and Atelectasis.
This study evaluates the effect of counterfactual explanations on the interpretation of chest X-rays. We conduct a reader study with two radiologists assessing 240 chest X-ray predictions to rate their confidence that the model's prediction is correct using a 5 point scale. Half of the predictions are false positives. Each prediction is explained twice, once using traditional attribution methods and once with a counterfactual explanation. The overall results indicate that counterfactual explanations allow a radiologist to have more confidence in true positive predictions compared to traditional approaches (0.15$\pm$0.95 with p=0.01) with only a small increase in false positive predictions (0.04$\pm$1.06 with p=0.57). We observe the specific prediction tasks of Mass and Atelectasis appear to benefit the most compared to other tasks.