Bilingual End-to-End ASR with Byte-Level Subwords
This work addresses bilingual ASR for speakers who do not switch languages within utterances, offering incremental improvements in performance.
The paper tackled the problem of improving utterance-based bilingual automatic speech recognition by exploring different output representations, finding that byte-level byte pair encoding with penalty schemes improved performance by 2% to 5% relative on English and Mandarin tasks.
In this paper, we investigate how the output representation of an end-to-end neural network affects multilingual automatic speech recognition (ASR). We study different representations including character-level, byte-level, byte pair encoding (BPE), and byte-level byte pair encoding (BBPE) representations, and analyze their strengths and weaknesses. We focus on developing a single end-to-end model to support utterance-based bilingual ASR, where speakers do not alternate between two languages in a single utterance but may change languages across utterances. We conduct our experiments on English and Mandarin dictation tasks, and we find that BBPE with penalty schemes can improve utterance-based bilingual ASR performance by 2% to 5% relative even with smaller number of outputs and fewer parameters. We conclude with analysis that indicates directions for further improving multilingual ASR.