Improving Mandarin Speech Recogntion with Block-augmented Transformer
This work addresses speech recognition for Mandarin speakers by introducing an incremental improvement over existing Conformer-based models.
The paper tackles the problem of improving Mandarin speech recognition by leveraging complementary information from each block in Transformer encoders and decoders, proposing Blockformer with two ensemble methods, and achieves a character error rate (CER) of 4.29% without a language model and 4.05% with one on the AISHELL-1 testset.
Recently Convolution-augmented Transformer (Conformer) has shown promising results in Automatic Speech Recognition (ASR), outperforming the previous best published Transformer Transducer. In this work, we believe that the output information of each block in the encoder and decoder is not completely inclusive, in other words, their output information may be complementary. We study how to take advantage of the complementary information of each block in a parameter-efficient way, and it is expected that this may lead to more robust performance. Therefore we propose the Block-augmented Transformer for speech recognition, named Blockformer. We have implemented two block ensemble methods: the base Weighted Sum of the Blocks Output (Base-WSBO), and the Squeeze-and-Excitation module to Weighted Sum of the Blocks Output (SE-WSBO). Experiments have proved that the Blockformer significantly outperforms the state-of-the-art Conformer-based models on AISHELL-1, our model achieves a CER of 4.29\% without using a language model and 4.05\% with an external language model on the testset.