SSplain: Sparse and Smooth Explainer for Retinopathy of Prematurity Classification
This work addresses the need for better model explainability in medical imaging, specifically for Retinopathy of Prematurity classification, though it is incremental as it builds on prior explainer methods.
The paper tackles the problem of generating realistic explanations for neural networks in medical diagnosis by introducing SSplain, a method that enforces smoothness and sparsity to preserve image structures, resulting in improved post-hoc accuracy and smoothness analyses compared to existing explainers.
Neural networks are frequently used in medical diagnosis. However, due to their black-box nature, model explainers are used to help clinicians understand better and trust model outputs. This paper introduces an explainer method for classifying Retinopathy of Prematurity (ROP) from fundus images. Previous methods fail to generate explanations that preserve input image structures such as smoothness and sparsity. We introduce Sparse and Smooth Explainer (SSplain), a method that generates pixel-wise explanations while preserving image structures by enforcing smoothness and sparsity. This results in realistic explanations to enhance the understanding of the given black-box model. To achieve this goal, we define an optimization problem with combinatorial constraints and solve it using the Alternating Direction Method of Multipliers (ADMM). Experimental results show that SSplain outperforms commonly used explainers in terms of both post-hoc accuracy and smoothness analyses. Additionally, SSplain identifies features that are consistent with domain-understandable features that clinicians consider as discriminative factors for ROP. We also show SSplain's generalization by applying it to additional publicly available datasets. Code is available at https://github.com/neu-spiral/SSplain.