PCA- and SVM-Grad-CAM for Convolutional Neural Networks: Closed-form Jacobian Expression
This work addresses the need for white-box interpretability in CNNs when using PCA or SVM layers, particularly for limited training data scenarios, though it is incremental as it extends existing Grad-CAM techniques.
The paper tackled the problem of visualizing attention regions in CNNs with PCA or SVM layers, which traditional Grad-CAM cannot handle, by proposing PCA-Grad-CAM and SVM-Grad-CAM methods and deriving a closed-form Jacobian expression, with results demonstrated on several major datasets.
Convolutional Neural Networks (CNNs) are an effective approach for classification tasks, particularly when the training dataset is large. Although CNNs have long been considered a black-box classification method, they can be used as a white-box method through visualization techniques such as Grad-CAM. When training samples are limited, incorporating a Principal Component Analysis (PCA) layer and/or a Support Vector Machine (SVM) classifier into a CNN can effectively improve classification performance. However, traditional Grad-CAM cannot be directly applied to PCA and/or SVM layers. It is important to generate attention regions for PCA and/or SVM layers in CNNs to facilitate the development of white-box methods. Therefore, we propose ``PCA-Grad-CAM'', a method for visualizing attention regions in PCA feature vectors, and ``SVM-Grad-CAM'', a method for visualizing attention regions in an SVM classifier layer. To complete our methods analytically, it is necessary to solve the closed-form Jacobian consisting of partial derivatives from the last convolutional layer to the PCA and/or SVM layers. In this paper, we present the exact closed-form Jacobian and the visualization results of our methods applied to several major datasets.