Improved Training for End-to-End Streaming Automatic Speech Recognition Model with Punctuation
This work addresses the challenge of enhancing readability and downstream NLP tasks in streaming ASR, though it is incremental as it builds on existing CTC and Transformer methods.
The paper tackles the problem of real-time punctuation prediction in streaming automatic speech recognition by proposing a chunk-based Transformer encoder trained with CTC loss, achieving improved F1 scores for punctuation and Word Error Rate.
Punctuated text prediction is crucial for automatic speech recognition as it enhances readability and impacts downstream natural language processing tasks. In streaming scenarios, the ability to predict punctuation in real-time is particularly desirable but presents a difficult technical challenge. In this work, we propose a method for predicting punctuated text from input speech using a chunk-based Transformer encoder trained with Connectionist Temporal Classification (CTC) loss. The acoustic model trained with long sequences by concatenating the input and target sequences can learn punctuation marks attached to the end of sentences more effectively. Additionally, by combining CTC losses on the chunks and utterances, we achieved both the improved F1 score of punctuation prediction and Word Error Rate (WER).