SENS-ASR: Semantic Embedding injection in Neural-transducer for Streaming Automatic Speech Recognition
This work addresses the problem of low-latency streaming ASR for applications requiring real-time transcription, but it is incremental as it builds on existing neural-transducer methods.
The paper tackles the performance degradation in streaming automatic speech recognition due to limited future context by injecting semantic information from past embeddings, resulting in significant improvements in Word Error Rate on small-chunk streaming scenarios.
Many Automatic Speech Recognition (ASR) applications require streaming processing of the audio data. In streaming mode, ASR systems need to start transcribing the input stream before it is complete, i.e., the systems have to process a stream of inputs with a limited (or no) future context. Compared to offline mode, this reduction of the future context degrades the performance of Streaming-ASR systems, especially while working with low-latency constraint. In this work, we present SENS-ASR, an approach to enhance the transcription quality of Streaming-ASR by reinforcing the acoustic information with semantic information. This semantic information is extracted from the available past frame-embeddings by a context module. This module is trained using knowledge distillation from a sentence embedding Language Model fine-tuned on the training dataset transcriptions. Experiments on standard datasets show that SENS-ASR significantly improves the Word Error Rate on small-chunk streaming scenarios.