Evolution Strategy-Based Calibration for Low-Bit Quantization of Speech Models
This work is significant for researchers and practitioners deploying efficient speech processing systems, as it provides a method to achieve high-performance low-bit quantization for speech models, an area where existing methods developed for vision and NLP often fall short.
This paper addresses the challenge of low-bit quantization for speech models, where audio activations often have large calibration ranges causing significant information loss. The authors propose ESC, an Evolution Strategy-based Calibration method, which enables unaltered performance under full INT8 quantization and near-lossless performance for full INT4 quantization across multiple speech tasks. When integrated with PTQ methods, ESC further reduces performance loss, achieving a 1% relative accuracy degradation on the AST model.
Quantization has become essential for the efficient deployment of speech processing systems. Although widely studied, most existing quantization methods were developed for vision and NLP architectures, while the specific challenges of audio signals remain largely overlooked. In particular, we show that audio activations can exhibit large calibration ranges, leading to significant information loss when standard calibration techniques are applied. To address this, we propose ESC, an Evolution Strategy-based Calibration method that formulates activation scaling as an optimization problem and solves it using a two-step local-global scheme driven by an evolution strategy. ESC enables unaltered performance under full INT8 quantization and is the first calibration method to achieve near-lossless performance for full INT4 quantization across multiple speech tasks. Integrating ESC with PTQ methods further reduces performance loss, achieving a 1% relative accuracy degradation on the AST model.