Non-Autoregressive Minimum Bayes' Risk Decoding for Fast Speech Recognition
This work addresses the performance gap in non-autoregressive speech recognition, offering a faster alternative to autoregressive decoding without sacrificing accuracy.
Non-autoregressive (NAR) speech recognition suffers from degraded performance due to parallel decoding. The authors propose NAR-MBR decoding, which maximizes expected utility from multiple samples obtained in a single forward pass, outperforming previous NAR methods and running faster than autoregressive decoding on LibriSpeech, Switchboard, AMI, and web presentation corpus.
Non-autoregressive (NAR) decoding generates output tokens in parallel, making speech recognition faster than autoregressive decoding, which generates them sequentially from left to right. However, the recognition performance is degraded because NAR decoding cannot resolve uncertainty by conditioning on previously generated tokens. To address this issue, we propose a novel NAR decoding framework based on minimum Bayes' risk (MBR) decoding, termed NAR-MBR decoding, that maximizes the expected utility calculated from samples drawn from the output probability of an NAR model rather than maximizing the output probability. Notably, by leveraging the nature of NAR models, multiple samples are obtained efficiently with a single forward computation. Our experiments across LibriSpeech, Switchboard, AMI, and web presentation corpus demonstrated that our NAR-MBR decoding outperformed previous NAR decoding and ran faster than AR decoding.