QMamba: Post-Training Quantization for Vision State Space Models
This work addresses the computational cost of SSMs for vision tasks on edge devices, representing an incremental advancement in quantization methods for a specific domain.
The paper tackles the problem of efficiently deploying vision State Space Models (SSMs) on resource-limited edge devices by proposing QMamba, a post-training quantization framework that reduces quantization errors through specialized techniques for discrete parameters and hidden states, achieving a 21.0% improvement on ImageNet classification with 4-bit activations.
State Space Models (SSMs), as key components of Mamaba, have gained increasing attention for vision models recently, thanks to their efficient long sequence modeling capability. Given the computational cost of deploying SSMs on resource-limited edge devices, Post-Training Quantization (PTQ) is a technique with the potential for efficient deployment of SSMs. In this work, we propose QMamba, one of the first PTQ frameworks to our knowledge, designed for vision SSMs based on the analysis of the activation distributions in SSMs. We reveal that the distribution of discrete parameters exhibits long-tailed skewness and the distribution of the hidden state sequence exhibits highly dynamic variations. Correspondingly, we design Long-tailed Skewness Quantization (LtSQ) to quantize discrete parameters and Temporal Group Quantization (TGQ) to quantize hidden states, which reduces the quantization errors. Extensive experiments demonstrate that QMamba outperforms advanced PTQ methods on vision models across multiple model sizes and architectures. Notably, QMamba surpasses existing methods by 21.0% on ImageNet classification with 4-bit activations.