Full-Sum Decoding for Hybrid HMM based Speech Recognition using LSTM Language Model
This work addresses incremental improvements in speech recognition for applications requiring high accuracy, such as transcription services.
The paper tackles the problem of improving speech recognition accuracy by using full-sum decoding instead of Viterbi approximation in hybrid HMM systems with LSTM language models, achieving consistent improvements on Switchboard and Librispeech corpora without extra cost.
In hybrid HMM based speech recognition, LSTM language models have been widely applied and achieved large improvements. The theoretical capability of modeling any unlimited context suggests that no recombination should be applied in decoding. This motivates to reconsider full summation over the HMM-state sequences instead of Viterbi approximation in decoding. We explore the potential gain from more accurate probabilities in terms of decision making and apply the full-sum decoding with a modified prefix-tree search framework. The proposed full-sum decoding is evaluated on both Switchboard and Librispeech corpora. Different models using CE and sMBR training criteria are used. Additionally, both MAP and confusion network decoding as approximated variants of general Bayes decision rule are evaluated. Consistent improvements over strong baselines are achieved in almost all cases without extra cost. We also discuss tuning effort, efficiency and some limitations of full-sum decoding.