Future Vector Enhanced LSTM Language Model for LVCSR
This addresses a specific bottleneck in LVCSR by enhancing sequence modeling, though it appears incremental as it builds on existing LSTM methods.
The paper tackles the mismatch between single-word prediction in traditional language models and the need for long-term sequence prediction in large vocabulary continuous speech recognition (LVCSR) by proposing an enhanced LSTM language model that incorporates future vectors to model sequence-level information. Experiments show it improves BLEU scores for long-term prediction and, when combined with conventional LSTM LMs in rescoring, achieves a very large improvement in word error rate.
Language models (LM) play an important role in large vocabulary continuous speech recognition (LVCSR). However, traditional language models only predict next single word with given history, while the consecutive predictions on a sequence of words are usually demanded and useful in LVCSR. The mismatch between the single word prediction modeling in trained and the long term sequence prediction in read demands may lead to the performance degradation. In this paper, a novel enhanced long short-term memory (LSTM) LM using the future vector is proposed. In addition to the given history, the rest of the sequence will be also embedded by future vectors. This future vector can be incorporated with the LSTM LM, so it has the ability to model much longer term sequence level information. Experiments show that, the proposed new LSTM LM gets a better result on BLEU scores for long term sequence prediction. For the speech recognition rescoring, although the proposed LSTM LM obtains very slight gains, the new model seems obtain the great complementary with the conventional LSTM LM. Rescoring using both the new and conventional LSTM LMs can achieve a very large improvement on the word error rate.