Exploring SSL Discrete Tokens for Multilingual ASR
It addresses the problem of adapting discrete tokens for multilingual ASR, which is incremental as it extends existing methods to new scenarios.
This study tackled the gap in using self-supervised learning (SSL) discrete tokens for multilingual automatic speech recognition (ASR), finding that discrete tokens achieve comparable performance to Fbank features with an average word error rate reduction of 0.31% and 1.76% absolute on dev and test sets across seven languages.
With the advancement of Self-supervised Learning (SSL) in speech-related tasks, there has been growing interest in utilizing discrete tokens generated by SSL for automatic speech recognition (ASR), as they offer faster processing techniques. However, previous studies primarily focused on multilingual ASR with Fbank features or English ASR with discrete tokens, leaving a gap in adapting discrete tokens for multilingual ASR scenarios. This study presents a comprehensive comparison of discrete tokens generated by various leading SSL models across multiple language domains. We aim to explore the performance and efficiency of speech discrete tokens across multiple language domains for both monolingual and multilingual ASR scenarios. Experimental results demonstrate that discrete tokens achieve comparable results against systems trained on Fbank features in ASR tasks across seven language domains with an average word error rate (WER) reduction of 0.31% and 1.76% absolute (2.80% and 15.70% relative) on dev and test sets respectively, with particularly WER reduction of 6.82% absolute (41.48% relative) on the Polish test set.