Differentiable Joint Pruning and Quantization for Hardware Efficiency
This work addresses the problem of hardware-efficient neural network compression for AI practitioners, offering an incremental improvement over existing two-stage methods by integrating pruning and quantization into a single optimization process.
The paper tackles neural network compression by proposing a differentiable joint pruning and quantization scheme that automatically trades off between pruning and quantization for hardware efficiency, achieving significant reductions in Bit-Operations (e.g., 53x in ResNet18 on ImageNet) while maintaining top-1 accuracy of original models.
We present a differentiable joint pruning and quantization (DJPQ) scheme. We frame neural network compression as a joint gradient-based optimization problem, trading off between model pruning and quantization automatically for hardware efficiency. DJPQ incorporates variational information bottleneck based structured pruning and mixed-bit precision quantization into a single differentiable loss function. In contrast to previous works which consider pruning and quantization separately, our method enables users to find the optimal trade-off between both in a single training procedure. To utilize the method for more efficient hardware inference, we extend DJPQ to integrate structured pruning with power-of-two bit-restricted quantization. We show that DJPQ significantly reduces the number of Bit-Operations (BOPs) for several networks while maintaining the top-1 accuracy of original floating-point models (e.g., 53x BOPs reduction in ResNet18 on ImageNet, 43x in MobileNetV2). Compared to the conventional two-stage approach, which optimizes pruning and quantization independently, our scheme outperforms in terms of both accuracy and BOPs. Even when considering bit-restricted quantization, DJPQ achieves larger compression ratios and better accuracy than the two-stage approach.