Explainability in CNN Models By Means of Z-Scores
This provides a simple framework for explainability in domain-specific applications like remote sensing, but it is incremental as it adapts existing statistical methods to neural networks.
The paper tackled the problem of explaining input importance in CNN models by using Z-scores derived from similarities with logistic regression, applied to a network for fusing SAR and MWR data to predict arctic sea ice, finding that MWR components are more important than SAR.
This paper explores the similarities of output layers in Neural Networks (NNs) with logistic regression to explain importance of inputs by Z-scores. The network analyzed, a network for fusion of Synthetic Aperture Radar (SAR) and Microwave Radiometry (MWR) data, is applied to prediction of arctic sea ice. With the analysis the importance of MWR relative to SAR is found to favor MWR components. Further, as the model represents image features at different scales, the relative importance of these are as well analyzed. The suggested methodology offers a simple and easy framework for analyzing output layer components and can reduce the number of components for further analysis with e.g. common NN visualization methods.