EfficientQuant: An Efficient Post-Training Quantization for CNN-Transformer Hybrid Models on Edge Devices
This work addresses the resource-intensive nature of hybrid models for edge deployment, offering a practical solution for real-world applications, though it is incremental as it builds on existing PTQ techniques.
The paper tackled the challenge of deploying CNN-transformer hybrid models on edge devices by proposing EfficientQuant, a post-training quantization method that reduces latency by 2.5x to 8.7x on ImageNet-1K with minimal accuracy loss.
Hybrid models that combine convolutional and transformer blocks offer strong performance in computer vision (CV) tasks but are resource-intensive for edge deployment. Although post-training quantization (PTQ) can help reduce resource demand, its application to hybrid models remains limited. We propose EfficientQuant, a novel structure-aware PTQ approach that applies uniform quantization to convolutional blocks and $log_2$ quantization to transformer blocks. EfficientQuant achieves $2.5 \times - 8.7 \times$ latency reduction with minimal accuracy loss on the ImageNet-1K dataset. It further demonstrates low latency and memory efficiency on edge devices, making it practical for real-world deployment.