Internal Language Model Estimation Through Explicit Context Vector Learning for Attention-based Encoder-decoder ASR
This work addresses a bottleneck in ASR systems for improving speech recognition accuracy by refining language model integration, though it is incremental as it builds on existing LAS frameworks.
The paper tackles the problem of accurately estimating the internal language model (ILM) in attention-based encoder-decoder ASR to enable effective fusion with external language models, achieving the lowest perplexity and outperforming prior methods like shallow fusion and previous ILM estimation approaches on several datasets.
An end-to-end (E2E) ASR model implicitly learns a prior Internal Language Model (ILM) from the training transcripts. To fuse an external LM using Bayes posterior theory, the log likelihood produced by the ILM has to be accurately estimated and subtracted. In this paper we propose two novel approaches to estimate the ILM based on Listen-Attend-Spell (LAS) framework. The first method is to replace the context vector of the LAS decoder at every time step with a vector that is learned with training transcripts. Furthermore, we propose another method that uses a lightweight feed-forward network to directly map query vector to context vector in a dynamic sense. Since the context vectors are learned by minimizing the perplexities on training transcripts, and their estimation is independent of encoder output, hence the ILMs are accurately learned for both methods. Experiments show that the ILMs achieve the lowest perplexity, indicating the efficacy of the proposed methods. In addition, they also significantly outperform the shallow fusion method, as well as two previously proposed ILM Estimation (ILME) approaches on several datasets.