Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition
This addresses the challenge of capturing pronunciation features in Mandarin ASR, which is incremental as it builds on existing encoder-decoder frameworks with a novel unit integration approach.
The paper tackles the problem of Mandarin speech recognition by proposing a multi-level modeling units method that integrates syllable and character information, achieving character error rates (CER) of 4.1%/4.6% and 4.6%/5.2% on the AISHELL-1 corpus with Conformer and Transformer backbones.
The choice of modeling units is crucial for automatic speech recognition (ASR) tasks. In mandarin scenarios, the Chinese characters represent meaning but are not directly related to the pronunciation. Thus only considering the writing of Chinese characters as modeling units is insufficient to capture speech features. In this paper, we present a novel method involves with multi-level modeling units, which integrates multi-level information for mandarin speech recognition. Specifically, the encoder block considers syllables as modeling units and the decoder block deals with character-level modeling units. To facilitate the incremental conversion from syllable features to character features, we design an auxiliary task that applies cross-entropy (CE) loss to intermediate decoder layers. During inference, the input feature sequences are converted into syllable sequences by the encoder block and then converted into Chinese characters by the decoder block. Experiments on the widely used AISHELL-1 corpus demonstrate that our method achieves promising results with CER of 4.1%/4.6% and 4.6%/5.2%, using the Conformer and the Transformer backbones respectively.