Robust Neural Pruning with Gradient Sampling Optimization for Residual Neural Networks
This work addresses the problem of efficient neural network deployment in limited-resource environments, but it appears incremental as it builds on existing pruning and optimization techniques.
The paper tackles the challenge of maintaining accuracy in pruned neural networks for resource-constrained scenarios by integrating gradient sampling optimization (StochGradAdam) into pruning, demonstrating that it significantly preserves accuracy compared to traditional methods, with results validated on CIFAR-10 datasets and residual architectures.
This research embarks on pioneering the integration of gradient sampling optimization techniques, particularly StochGradAdam, into the pruning process of neural networks. Our main objective is to address the significant challenge of maintaining accuracy in pruned neural models, critical in resource-constrained scenarios. Through extensive experimentation, we demonstrate that gradient sampling significantly preserves accuracy during and after the pruning process compared to traditional optimization methods. Our study highlights the pivotal role of gradient sampling in robust learning and maintaining crucial information post substantial model simplification. The results across CIFAR-10 datasets and residual neural architectures validate the versatility and effectiveness of our approach. This work presents a promising direction for developing efficient neural networks without compromising performance, even in environments with limited computational resources.