Architecture Aware Latency Constrained Sparse Neural Networks
This work addresses the need for efficient deployment of CNNs on resource-constrained mobile devices, representing an incremental improvement in system-algorithm co-design.
The paper tackled the problem of accelerating deep neural networks to meet latency constraints on mobile devices by proposing an architecture-aware sparse neural network framework, achieving a better Pareto frontier between accuracy and latency.
Acceleration of deep neural networks to meet a specific latency constraint is essential for their deployment on mobile devices. In this paper, we design an architecture aware latency constrained sparse (ALCS) framework to prune and accelerate CNN models. Taking modern mobile computation architectures into consideration, we propose Single Instruction Multiple Data (SIMD)-structured pruning, along with a novel sparse convolution algorithm for efficient computation. Besides, we propose to estimate the run time of sparse models with piece-wise linear interpolation. The whole latency constrained pruning task is formulated as a constrained optimization problem that can be efficiently solved with Alternating Direction Method of Multipliers (ADMM). Extensive experiments show that our system-algorithm co-design framework can achieve much better Pareto frontier among network accuracy and latency on resource-constrained mobile devices.