Beyond One-Way Pruning: Bidirectional Pruning-Regrowth for Extreme Accuracy-Sparsity Tradeoff
This addresses the challenge of deploying compressed models on hardware with strict size limits, though it appears incremental as it builds on existing pruning techniques.
The paper tackles the problem of severe performance degradation in model pruning at high sparsity levels, which limits compression ratios for hardware constraints, by proposing a bidirectional pruning-regrowth strategy that recovers accuracy by regenerating critical connections, achieving improved tradeoffs without specific numbers provided.
As a widely adopted model compression technique, model pruning has demonstrated strong effectiveness across various architectures. However, we observe that when sparsity exceeds a certain threshold, both iterative and one-shot pruning methods lead to a steep decline in model performance. This rapid degradation limits the achievable compression ratio and prevents models from meeting the stringent size constraints required by certain hardware platforms, rendering them inoperable. To overcome this limitation, we propose a bidirectional pruning-regrowth strategy. Starting from an extremely compressed network that satisfies hardware constraints, the method selectively regenerates critical connections to recover lost performance, effectively mitigating the sharp accuracy drop commonly observed under high sparsity conditions.