Pruning for Feature-Preserving Circuits in CNNs
This work addresses interpretability for researchers and practitioners in computer vision, but it is incremental as it builds on existing pruning methods.
The authors tackled the problem of interpreting deep convolutional neural networks by introducing a method to extract 'feature-preserving circuits' using saliency-based pruning, resulting in modular sub-functions that isolate relevant kernels for target features and enable visualization of the filtering process.
Deep convolutional neural networks are a powerful model class for a range of computer vision problems, but it is difficult to interpret the image filtering process they implement, given their sheer size. In this work, we introduce a method for extracting 'feature-preserving circuits' from deep CNNs, leveraging methods from saliency-based neural network pruning. These circuits are modular sub-functions, embedded within the network, containing only a subset of convolutional kernels relevant to a target feature. We compare the efficacy of 3 saliency-criteria for extracting these sparse circuits. Further, we show how 'sub-feature' circuits can be extracted, that preserve a feature's responses to particular images, dividing the feature into even sparser filtering processes. We also develop a tool for visualizing 'circuit diagrams', which render the entire image filtering process implemented by circuits in a parsable format.