Adaptive Discounting of Implicit Language Models in RNN-Transducers
This addresses rare word errors in streaming ASR systems, particularly for conversational code-mixed speech, with an incremental improvement to existing RNN-T architectures.
The paper tackles the problem of rare word recognition degradation in RNN-Transducer ASR systems by proposing AdaptLMD, a lightweight adaptive language model discounting technique, resulting in up to 4% relative reduction in overall WER and 14% in rare word PER on a Hindi-English ASR task.
RNN-Transducer (RNN-T) models have become synonymous with streaming end-to-end ASR systems. While they perform competitively on a number of evaluation categories, rare words pose a serious challenge to RNN-T models. One main reason for the degradation in performance on rare words is that the language model (LM) internal to RNN-Ts can become overconfident and lead to hallucinated predictions that are acoustically inconsistent with the underlying speech. To address this issue, we propose a lightweight adaptive LM discounting technique AdaptLMD, that can be used with any RNN-T architecture without requiring any external resources or additional parameters. AdaptLMD uses a two-pronged approach: 1) Randomly mask the prediction network output to encourage the RNN-T to not be overly reliant on it's outputs. 2) Dynamically choose when to discount the implicit LM (ILM) based on rarity of recently predicted tokens and divergence between ILM and implicit acoustic model (IAM) scores. Comparing AdaptLMD to a competitive RNN-T baseline, we obtain up to 4% and 14% relative reductions in overall WER and rare word PER, respectively, on a conversational, code-mixed Hindi-English ASR task.