Soft-CAM: Making black box models self-explainable for high-stakes decisions
This addresses the need for trustworthy self-explainable models in critical domains like medicine, though it is incremental as it modifies existing CNN architectures.
The paper tackled the problem of unreliable post-hoc explanation methods for CNNs in high-stakes applications by introducing SoftCAM, which makes CNNs inherently interpretable while maintaining classification performance on three medical datasets, with significant improvements in explanation quality.
Convolutional neural networks (CNNs) are widely used for high-stakes applications like medicine, often surpassing human performance. However, most explanation methods rely on post-hoc attribution, approximating the decision-making process of already trained black-box models. These methods are often sensitive, unreliable, and fail to reflect true model reasoning, limiting their trustworthiness in critical applications. In this work, we introduce SoftCAM, a straightforward yet effective approach that makes standard CNN architectures inherently interpretable. By removing the global average pooling layer and replacing the fully connected classification layer with a convolution-based class evidence layer, SoftCAM preserves spatial information and produces explicit class activation maps that form the basis of the model's predictions. Evaluated on three medical datasets, SoftCAM maintains classification performance while significantly improving both the qualitative and quantitative explanation compared to existing post-hoc methods. Our results demonstrate that CNNs can be inherently interpretable without compromising performance, advancing the development of self-explainable deep learning for high-stakes decision-making.