Convolutional Neural Network Interpretability with General Pattern Theory
This work aims to improve the interpretability of CNNs for researchers and practitioners by reducing the impact of dataset complexity on understanding model decisions, representing an incremental step in XAI research.
This paper proposes using Ulf Grenander's pattern theory to interpret Convolutional Neural Networks (CNNs) by describing data as configurations of fundamental objects. They designed a U-Net-like architecture with expansion blocks attached to ResNet, enabling semantic segmentation-like tasks compatible with pattern theory's configurations, which allows heatmap-based XAI methods to extract explanations for individual data sample generators.
Ongoing efforts to understand deep neural networks (DNN) have provided many insights, but DNNs remain incompletely understood. Improving DNN's interpretability has practical benefits, such as more accountable usage, better algorithm maintenance and improvement. The complexity of dataset structure may contribute to the difficulty in solving interpretability problem arising from DNN's black-box mechanism. Thus, we propose to use pattern theory formulated by Ulf Grenander, in which data can be described as configurations of fundamental objects that allow us to investigate convolutional neural network's (CNN) interpretability in a component-wise manner. Specifically, U-Net-like structure is formed by attaching expansion blocks (EB) to ResNet, allowing it to perform semantic segmentation-like tasks at its EB output channels designed to be compatible with pattern theory's configurations. Through these modules, some heatmap-based explainable artificial intelligence (XAI) methods will be shown to extract explanations w.r.t individual generators that make up a single data sample, potentially reducing the impact of dataset's complexity to interpretability problem. The MNIST-equivalent dataset containing pattern theory's elements is designed to facilitate smoother entry into this framework, along which the theory's generative aspect is naturally presented.