Learning not to Discriminate: Task Agnostic Learning for Improving Monolingual and Code-switched Speech Recognition
This addresses the challenge of maintaining high accuracy in both monolingual and code-switched speech for ASR systems in multilingual scenarios, representing an incremental improvement over previous work.
The paper tackles the problem of automatic speech recognition (ASR) systems degrading in monolingual performance when fine-tuned on code-switched data, proposing domain adversarial learning to train task-agnostic models that improve both monolingual and code-switched recognition, leading to reductions in Word Error Rates (WER) across three language pairs.
Recognizing code-switched speech is challenging for Automatic Speech Recognition (ASR) for a variety of reasons, including the lack of code-switched training data. Recently, we showed that monolingual ASR systems fine-tuned on code-switched data deteriorate in performance on monolingual speech recognition, which is not desirable as ASR systems deployed in multilingual scenarios should recognize both monolingual and code-switched speech with high accuracy. Our experiments indicated that this loss in performance could be mitigated by using certain strategies for fine-tuning and regularization, leading to improvements in both monolingual and code-switched ASR. In this work, we present further improvements over our previous work by using domain adversarial learning to train task agnostic models. We evaluate the classification accuracy of an adversarial discriminator and show that it can learn shared layer parameters that are task agnostic. We train end-to-end ASR systems starting with a pooled model that uses monolingual and code-switched data along with the adversarial discriminator. Our proposed technique leads to reductions in Word Error Rates (WER) in monolingual and code-switched test sets across three language pairs.