Discriminative training of RNNLMs with the average word error criterion
This work addresses a specific problem in ASR by improving language model performance for lattice rescoring, though it is incremental as it builds on existing RNNLM methods.
The paper tackled the mismatch between generative training of recurrent neural language models (RNNLMs) using perplexity and the goal of minimizing word error rate (WER) in automatic speech recognition by proposing discriminative training with an average edit distance loss. This approach achieved a 1.9% relative improvement in WER over generatively trained models.
In automatic speech recognition (ASR), recurrent neural language models (RNNLM) are typically used to refine hypotheses in the form of lattices or n-best lists, which are generated by a beam search decoder with a weaker language model. The RNNLMs are usually trained generatively using the perplexity (PPL) criterion on large corpora of grammatically correct text. However, the hypotheses are noisy, and the RNNLM doesn't always make the choices that minimise the metric we optimise for, the word error rate (WER). To address this mismatch we propose to use a task specific loss to train an RNNLM to discriminate between multiple hypotheses within lattice rescoring scenario. By fine-tuning the RNNLM on lattices with the average edit distance loss, we show that we obtain a 1.9% relative improvement in word error rate over a purely generatively trained model.