Improving Explainability of Disentangled Representations using Multipath-Attribution Mappings
This work addresses the need for better explainability in safety-critical medical imaging systems, offering an incremental improvement over existing attribution methods.
The paper tackles the problem of explaining why visual features are used in AI models for image-based clinical diagnostics, proposing a framework that uses interpretable disentangled representations and multi-path attribution mapping, and demonstrates effectiveness on synthetic and medical datasets by enhancing model robustness and enabling shortcut detection.
Explainable AI aims to render model behavior understandable by humans, which can be seen as an intermediate step in extracting causal relations from correlative patterns. Due to the high risk of possible fatal decisions in image-based clinical diagnostics, it is necessary to integrate explainable AI into these safety-critical systems. Current explanatory methods typically assign attribution scores to pixel regions in the input image, indicating their importance for a model's decision. However, they fall short when explaining why a visual feature is used. We propose a framework that utilizes interpretable disentangled representations for downstream-task prediction. Through visualizing the disentangled representations, we enable experts to investigate possible causation effects by leveraging their domain knowledge. Additionally, we deploy a multi-path attribution mapping for enriching and validating explanations. We demonstrate the effectiveness of our approach on a synthetic benchmark suite and two medical datasets. We show that the framework not only acts as a catalyst for causal relation extraction but also enhances model robustness by enabling shortcut detection without the need for testing under distribution shifts.