What Do Compressed Deep Neural Networks Forget?
This work highlights the hidden biases and trade-offs in compressed models, which is critical for their safe deployment in real-world applications, though it is incremental in analyzing existing compression methods.
The paper investigates how deep neural network compression techniques, such as pruning and quantization, affect model behavior beyond overall accuracy, revealing that compression disproportionately impacts performance on a small subset of challenging, atypical images, termed Pruning Identified Exemplars (PIEs), which are often from underrepresented data distributions.
Deep neural network pruning and quantization techniques have demonstrated it is possible to achieve high levels of compression with surprisingly little degradation to test set accuracy. However, this measure of performance conceals significant differences in how different classes and images are impacted by model compression techniques. We find that models with radically different numbers of weights have comparable top-line performance metrics but diverge considerably in behavior on a narrow subset of the dataset. This small subset of data points, which we term Pruning Identified Exemplars (PIEs) are systematically more impacted by the introduction of sparsity. Compression disproportionately impacts model performance on the underrepresented long-tail of the data distribution. PIEs over-index on atypical or noisy images that are far more challenging for both humans and algorithms to classify. Our work provides intuition into the role of capacity in deep neural networks and the trade-offs incurred by compression. An understanding of this disparate impact is critical given the widespread deployment of compressed models in the wild.