OMPQ: Orthogonal Mixed Precision Quantization
This addresses the efficiency bottleneck in deploying complex neural networks on hardware with limited resources, offering a faster and less data-dependent solution for quantization.
The paper tackles the challenge of mixed precision quantization for deep neural networks by proposing a proxy metric based on network orthogonality, which reduces search time and data requirements by orders of magnitude while achieving 72.08% Top-1 accuracy on ResNet-18 with 6.7Mb and 71.27% on MobileNetV2 with 1.5Mb.
To bridge the ever increasing gap between deep neural networks' complexity and hardware capability, network quantization has attracted more and more research attention. The latest trend of mixed precision quantization takes advantage of hardware's multiple bit-width arithmetic operations to unleash the full potential of network quantization. However, this also results in a difficult integer programming formulation, and forces most existing approaches to use an extremely time-consuming search process even with various relaxations. Instead of solving a problem of the original integer programming, we propose to optimize a proxy metric, the concept of network orthogonality, which is highly correlated with the loss of the integer programming but also easy to optimize with linear programming. This approach reduces the search time and required data amount by orders of magnitude, with little compromise on quantization accuracy. Specifically, we achieve 72.08% Top-1 accuracy on ResNet-18 with 6.7Mb, which does not require any searching iterations. Given the high efficiency and low data dependency of our algorithm, we used it for the post-training quantization, which achieve 71.27% Top-1 accuracy on MobileNetV2 with only 1.5Mb. Our code is available at https://github.com/MAC-AutoML/OMPQ.