CTC-synchronous Training for Monotonic Attention Model
This addresses error propagation in online streaming ASR for applications requiring real-time processing, though it is incremental as it builds on existing MoChA and CTC methods.
The paper tackled the problem of error propagation in monotonic chunkwise attention (MoChA) for online streaming ASR by proposing CTC-synchronous training (CTC-ST), which uses CTC alignments to guide MoChA's monotonic alignments, resulting in significant recognition improvements, especially for long utterances on TEDLIUM-2 and Librispeech corpora.
Monotonic chunkwise attention (MoChA) has been studied for the online streaming automatic speech recognition (ASR) based on a sequence-to-sequence framework. In contrast to connectionist temporal classification (CTC), backward probabilities cannot be leveraged in the alignment marginalization process during training due to left-to-right dependency in the decoder. This results in the error propagation of alignments to subsequent token generation. To address this problem, we propose CTC-synchronous training (CTC-ST), in which MoChA uses CTC alignments to learn optimal monotonic alignments. Reference CTC alignments are extracted from a CTC branch sharing the same encoder with the decoder. The entire model is jointly optimized so that the expected boundaries from MoChA are synchronized with the alignments. Experimental evaluations of the TEDLIUM release-2 and Librispeech corpora show that the proposed method significantly improves recognition, especially for long utterances. We also show that CTC-ST can bring out the full potential of SpecAugment for MoChA.