RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging
This work addresses the need for efficient neural network deployment in resource-constrained environments by providing a data-free pruning solution, though it is incremental as it builds on existing pruning paradigms.
The paper tackles the problem of accelerating deep neural network inference by introducing RED++, a data-free pruning method that removes redundant operations without requiring original training data. The method achieves competitive pruning ratios and maintains accuracy, outperforming other data-free methods and matching data-driven approaches on models like ResNets and MobileNets.
Pruning Deep Neural Networks (DNNs) is a prominent field of study in the goal of inference runtime acceleration. In this paper, we introduce a novel data-free pruning protocol RED++. Only requiring a trained neural network, and not specific to DNN architecture, we exploit an adaptive data-free scalar hashing which exhibits redundancies among neuron weight values. We study the theoretical and empirical guarantees on the preservation of the accuracy from the hashing as well as the expected pruning ratio resulting from the exploitation of said redundancies. We propose a novel data-free pruning technique of DNN layers which removes the input-wise redundant operations. This algorithm is straightforward, parallelizable and offers novel perspective on DNN pruning by shifting the burden of large computation to efficient memory access and allocation. We provide theoretical guarantees on RED++ performance and empirically demonstrate its superiority over other data-free pruning methods and its competitiveness with data-driven ones on ResNets, MobileNets and EfficientNets.