DYNAC: Dynamic Vocabulary based Non-Autoregressive Contextualization for Speech Recognition
This work addresses the trade-off between accuracy and speed in speech recognition for rare phrases, offering a practical improvement for real-time applications.
The paper tackled the problem of slow inference speed in contextual biasing for speech recognition by proposing DYNAC, a non-autoregressive method that integrates dynamic vocabulary into intermediate layers, reducing real-time factor by 81% with only a 0.1-point degradation in word error rate on the LibriSpeech test-clean set.
Contextual biasing (CB) improves automatic speech recognition for rare and unseen phrases. Recent studies have introduced dynamic vocabulary, which represents context phrases as expandable tokens in autoregressive (AR) models. This method improves CB accuracy but with slow inference speed. While dynamic vocabulary can be applied to non-autoregressive (NAR) models, such as connectionist temporal classification (CTC), the conditional independence assumption fails to capture dependencies between static and dynamic tokens. This paper proposes DYNAC (Dynamic Vocabulary-based NAR Contextualization), a self-conditioned CTC method that integrates dynamic vocabulary into intermediate layers. Conditioning the encoder on dynamic vocabulary, DYNAC effectively captures dependencies between static and dynamic tokens while reducing the real-time factor (RTF). Experimental results show that DYNAC reduces RTF by 81% with a 0.1-point degradation in word error rate on the LibriSpeech 960 test-clean set.