Quantization for Rapid Deployment of Deep Neural Networks
This addresses the challenge of rapid deployment for AI practitioners by enabling quantization with minimal data, though it is incremental as it builds on existing quantization techniques.
The paper tackles the problem of deploying deep neural networks to energy-efficient accelerators without fine-tuning or full datasets by proposing a channel-level distribution method for quantization, achieving 8-bit integer precision on ImageNet and Pascal VOC benchmarks without accuracy loss.
This paper aims at rapid deployment of the state-of-the-art deep neural networks (DNNs) to energy efficient accelerators without time-consuming fine tuning or the availability of the full datasets. Converting DNNs in full precision to limited precision is essential in taking advantage of the accelerators with reduced memory footprint and computation power. However, such a task is not trivial since it often requires the full training and validation datasets for profiling the network statistics and fine tuning the networks to recover the accuracy lost after quantization. To address these issues, we propose a simple method recognizing channel-level distribution to reduce the quantization-induced accuracy loss and minimize the required image samples for profiling. We evaluated our method on eleven networks trained on the ImageNet classification benchmark and a network trained on the Pascal VOC object detection benchmark. The results prove that the networks can be quantized into 8-bit integer precision without fine tuning.