Structured Sparsity and Weight-adaptive Pruning for Memory and Compute efficient Whisper models
This work addresses the deployment bottleneck of speech recognition models on edge devices, representing an incremental improvement in model compression techniques.
The paper tackles the problem of deploying large Whisper models on resource-constrained edge devices by proposing a framework that uses structured sparsity and weight-adaptive pruning to reduce model size and computational costs, achieving up to 35.4% parameter reduction and 18.5% fewer FLOPs without degrading WER on the Common Voice 11.0 Hindi dataset.
Whisper models have achieved remarkable progress in speech recognition; yet their large size remains a bottleneck for deployment on resource-constrained edge devices. This paper proposes a framework to design fine-tuned variants of Whisper which address the above problem. Structured sparsity is enforced via the Sparse Group LASSO penalty as a loss regularizer, to reduce the number of FLOating Point operations (FLOPs). Further, a weight statistics aware pruning algorithm is proposed. We also design our custom text normalizer for WER evaluation. On Common Voice 11.0 Hindi dataset, we obtain, without degrading WER, (a) 35.4% reduction in model parameters, 14.25% lower memory consumption and 18.5% fewer FLOPs on Whisper-small, and (b) 31% reduction in model parameters, 15.29% lower memory consumption and 16.95% fewer FLOPs on Whisper-medium; and, (c) substantially outperform the state-of-the-art Iterative Magnitude Pruning based method by pruning 18.7% more parameters along with a 12.31 reduction in WER.