Unified model for code-switching speech recognition and language identification based on a concatenated tokenizer
This addresses the challenge of transcribing multilingual conversations for speech recognition systems, though it is incremental as it builds on existing monolingual tokenizers.
The paper tackles the problem of building code-switching automatic speech recognition models by proposing a method to create datasets from monolingual data and a concatenated tokenizer for language identification, achieving state-of-the-art results on English-Hindi and English-Spanish pairs and over 98% accuracy on language identification.
Code-Switching (CS) multilingual Automatic Speech Recognition (ASR) models can transcribe speech containing two or more alternating languages during a conversation. This paper proposes (1) a new method for creating code-switching ASR datasets from purely monolingual data sources, and (2) a novel Concatenated Tokenizer that enables ASR models to generate language ID for each emitted text token while reusing existing monolingual tokenizers. The efficacy of these approaches for building CS ASR models is demonstrated for two language pairs, English-Hindi and English-Spanish, where we achieve new state-of-the-art results on the Miami Bangor CS evaluation corpus. In addition to competitive ASR performance, the proposed Concatenated Tokenizer models are highly effective for spoken language identification, achieving 98%+ accuracy on the out-of-distribution FLEURS dataset.