Deadwooding: Robust Global Pruning for Deep Neural Networks
This addresses the deployment challenge of large models in resource-constrained environments by improving pruning techniques to retain accuracy and robustness, representing a strong specific gain rather than a foundational advancement.
The paper tackles the problem of pruning deep neural networks to reduce model size without sacrificing accuracy and adversarial robustness, introducing Deadwooding, a global pruning technique that uses a Lagrangian Dual method to achieve sparsity while maintaining performance, resulting in significant outperformance over state-of-the-art methods in robustness and accuracy.
The ability of Deep Neural Networks to approximate highly complex functions is key to their success. This benefit, however, comes at the expense of a large model size, which challenges its deployment in resource-constrained environments. Pruning is an effective technique used to limit this issue, but often comes at the cost of reduced accuracy and adversarial robustness. This paper addresses these shortcomings and introduces Deadwooding, a novel global pruning technique that exploits a Lagrangian Dual method to encourage model sparsity while retaining accuracy and ensuring robustness. The resulting model is shown to significantly outperform the state-of-the-art studies in measures of robustness and accuracy.