Dynamic latency speech recognition with asynchronous revision
This work addresses latency-accuracy trade-offs in speech recognition, which is incremental as it builds on existing models with a new inference technique.
The authors tackled the problem of unifying streaming and non-streaming speech recognition models to achieve dynamic latency, resulting in 8%-14% relative accuracy improvements over streaming models.
In this work we propose an inference technique, asynchronous revision, to unify streaming and non-streaming speech recognition models. Specifically, we achieve dynamic latency with only one model by using arbitrary right context during inference. The model is composed of a stack of convolutional layers for audio encoding. In inference stage, the history states of encoder and decoder can be asynchronously revised to trade off between the latency and the accuracy of the model. To alleviate training and inference mismatch, we propose a training technique, segment cropping, which randomly splits input utterances into several segments with forward connections. This allows us to have dynamic latency speech recognition results with large improvements in accuracy. Experiments show that our dynamic latency model with asynchronous revision gives 8\%-14\% relative improvements over the streaming models.