Fast On-the-fly Retraining-free Sparsification of Convolutional Neural Networks
This addresses the deployment challenges of CNNs on embedded platforms by providing a retraining-free solution, which is incremental as it builds on existing sparsification approaches.
The paper tackles the problem of reducing computational and storage demands of Convolutional Neural Networks (CNNs) by proposing on-the-fly sparsification methods that require no retraining, achieving up to 73% weight reduction with less than 5% loss in Top-5 accuracy.
Modern Convolutional Neural Networks (CNNs) are complex, encompassing millions of parameters. Their deployment exerts computational, storage and energy demands, particularly on embedded platforms. Existing approaches to prune or sparsify CNNs require retraining to maintain inference accuracy. Such retraining is not feasible in some contexts. In this paper, we explore the sparsification of CNNs by proposing three model-independent methods. Our methods are applied on-the-fly and require no retraining. We show that the state-of-the-art models' weights can be reduced by up to 73% (compression factor of 3.7x) without incurring more than 5% loss in Top-5 accuracy. Additional fine-tuning gains only 8% in sparsity, which indicates that our fast on-the-fly methods are effective.