Generalizable Mixed-Precision Quantization via Attribution Rank Preservation
This addresses the problem of heavy computational costs in quantization for researchers and practitioners in machine learning, though it is incremental as it builds on existing mixed-precision methods.
The paper tackles the high search cost of mixed-precision quantization for efficient inference by proposing a method that generalizes to large-scale datasets using only a small amount of data, achieving competitive accuracy-complexity trade-offs with significantly reduced search time.
In this paper, we propose a generalizable mixed-precision quantization (GMPQ) method for efficient inference. Conventional methods require the consistency of datasets for bitwidth search and model deployment to guarantee the policy optimality, leading to heavy search cost on challenging largescale datasets in realistic applications. On the contrary, our GMPQ searches the mixed-quantization policy that can be generalized to largescale datasets with only a small amount of data, so that the search cost is significantly reduced without performance degradation. Specifically, we observe that locating network attribution correctly is general ability for accurate visual analysis across different data distribution. Therefore, despite of pursuing higher model accuracy and complexity, we preserve attribution rank consistency between the quantized models and their full-precision counterparts via efficient capacity-aware attribution imitation for generalizable mixed-precision quantization strategy search. Extensive experiments show that our method obtains competitive accuracy-complexity trade-off compared with the state-of-the-art mixed-precision networks in significantly reduced search cost. The code is available at https://github.com/ZiweiWangTHU/GMPQ.git.