Dynamic Quantization Error Propagation in Encoder-Decoder ASR Quantization
This addresses quantization challenges for ASR on memory-constrained devices, but it is incremental as it builds on existing error propagation methods.
The paper tackled error accumulation in quantizing encoder-decoder ASR models for edge devices by proposing FADE, which adaptively controls error correction, resulting in improved stability and reduced mean WER compared to baselines.
Running Automatic Speech Recognition (ASR) models on memory-constrained edge devices requires efficient compression. While layer-wise post-training quantization is effective, it suffers from error accumulation, especially in encoder-decoder architectures. Existing solutions like Quantization Error Propagation (QEP) are suboptimal for ASR due to the model's heterogeneity, processing acoustic features in the encoder while generating text in the decoder. To address this, we propose Fine-grained Alpha for Dynamic Quantization Error Propagation (FADE), which adaptively controls the trade-off between cross-layer error correction and local quantization. Experiments show that FADE significantly improves stability by reducing performance variance across runs, while simultaneously surpassing baselines in mean WER.