Dissecting Pruned Neural Networks
This addresses the problem of balancing model compression and interpretability for AI practitioners, but it is incremental as it builds on existing pruning and interpretability methods.
The paper investigates how pruning affects interpretability in neural networks, finding that pruning does not harm interpretability until accuracy declines, with ResNet-50 models maintaining interpretable concepts until over 90% of parameters are pruned.
Pruning is a standard technique for removing unnecessary structure from a neural network to reduce its storage footprint, computational demands, or energy consumption. Pruning can reduce the parameter-counts of many state-of-the-art neural networks by an order of magnitude without compromising accuracy, meaning these networks contain a vast amount of unnecessary structure. In this paper, we study the relationship between pruning and interpretability. Namely, we consider the effect of removing unnecessary structure on the number of hidden units that learn disentangled representations of human-recognizable concepts as identified by network dissection. We aim to evaluate how the interpretability of pruned neural networks changes as they are compressed. We find that pruning has no detrimental effect on this measure of interpretability until so few parameters remain that accuracy beings to drop. Resnet-50 models trained on ImageNet maintain the same number of interpretable concepts and units until more than 90% of parameters have been pruned.