Streaming End-to-End ASR based on Blockwise Non-Autoregressive Models
This work addresses the need for efficient, low-latency speech recognition systems, particularly for real-time applications, by incrementally adapting non-autoregressive models to streaming scenarios.
The paper tackles the problem of low-latency streaming automatic speech recognition by proposing a blockwise non-autoregressive model that processes audio in small blocks, reducing insertion and deletion errors at block edges with an overlapping decoding strategy. Experimental results show improved online ASR recognition in low-latency conditions compared to vanilla Mask-CTC and faster inference speed than autoregressive attention-based models.
Non-autoregressive (NAR) modeling has gained more and more attention in speech processing. With recent state-of-the-art attention-based automatic speech recognition (ASR) structure, NAR can realize promising real-time factor (RTF) improvement with only small degradation of accuracy compared to the autoregressive (AR) models. However, the recognition inference needs to wait for the completion of a full speech utterance, which limits their applications on low latency scenarios. To address this issue, we propose a novel end-to-end streaming NAR speech recognition system by combining blockwise-attention and connectionist temporal classification with mask-predict (Mask-CTC) NAR. During inference, the input audio is separated into small blocks and then processed in a blockwise streaming way. To address the insertion and deletion error at the edge of the output of each block, we apply an overlapping decoding strategy with a dynamic mapping trick that can produce more coherent sentences. Experimental results show that the proposed method improves online ASR recognition in low latency conditions compared to vanilla Mask-CTC. Moreover, it can achieve a much faster inference speed compared to the AR attention-based models. All of our codes will be publicly available at https://github.com/espnet/espnet.