On Causal Inference for Data-free Structured Pruning
This work addresses the challenge of making neural networks more efficient for broader deployment, though it appears incremental as it builds on existing pruning methods with a novel causal perspective.
The paper tackles the problem of neural network compression by proposing a causal inference-based scoring mechanism for structured pruning, achieving competitive performance on two datasets and various network sizes.
Neural networks (NNs) are making a large impact both on research and industry. Nevertheless, as NNs' accuracy increases, it is followed by an expansion in their size, required number of compute operations and energy consumption. Increase in resource consumption results in NNs' reduced adoption rate and real-world deployment impracticality. Therefore, NNs need to be compressed to make them available to a wider audience and at the same time decrease their runtime costs. In this work, we approach this challenge from a causal inference perspective, and we propose a scoring mechanism to facilitate structured pruning of NNs. The approach is based on measuring mutual information under a maximum entropy perturbation, sequentially propagated through the NN. We demonstrate the method's performance on two datasets and various NNs' sizes, and we show that our approach achieves competitive performance under challenging conditions.