A copula-based visualization technique for a neural network
This work addresses the interpretability issue in neural networks for researchers and practitioners, but it is incremental as it builds on existing visualization and interpretability methods.
The authors tackled the problem of neural network interpretability by proposing a new algorithm that identifies important features and decision paths using copula-based correlation scores, and they validated it by showing consistency with Random Forest feature importance.
Interpretability of machine learning is defined as the extent to which humans can comprehend the reason of a decision. However, a neural network is not considered interpretable due to the ambiguity in its decision-making process. Therefore, in this study, we propose a new algorithm that reveals which feature values the trained neural network considers important and which paths are mainly traced in the process of decision-making. In the proposed algorithm, the score estimated by the correlation coefficients between the neural network layers that can be calculated by applying the concept of a pair copula was defined. We compared the estimated score with the feature importance values of Random Forest, which is sometimes regarded as a highly interpretable algorithm, in the experiment and confirmed that the results were consistent with each other. This algorithm suggests an approach for compressing a neural network and its parameter tuning because the algorithm identifies the paths that contribute to the classification or prediction results.