Style Variation as a Vantage Point for Code-Switching
This addresses the data scarcity issue for code-switching modeling in bilingual and multilingual communities, enabling improved downstream tasks like speech recognition, though it is incremental in its method.
The paper tackles the problem of generating code-switching text without requiring annotated data by framing code-switching as style variations between languages, using a two-stage adversarial training approach on monolingual corpora and limited natural code-switching sentences. The result shows that generated code-switching metrics move closer to real data across multiple language pairs, such as Spanish-English and Mandarin-English.
Code-Switching (CS) is a common phenomenon observed in several bilingual and multilingual communities, thereby attaining prevalence in digital and social media platforms. This increasing prominence demands the need to model CS languages for critical downstream tasks. A major problem in this domain is the dearth of annotated data and a substantial corpora to train large scale neural models. Generating vast amounts of quality text assists several down stream tasks that heavily rely on language modeling such as speech recognition, text-to-speech synthesis etc,. We present a novel vantage point of CS to be style variations between both the participating languages. Our approach does not need any external annotations such as lexical language ids. It mainly relies on easily obtainable monolingual corpora without any parallel alignment and a limited set of naturally CS sentences. We propose a two-stage generative adversarial training approach where the first stage generates competitive negative examples for CS and the second stage generates more realistic CS sentences. We present our experiments on the following pairs of languages: Spanish-English, Mandarin-English, Hindi-English and Arabic-French. We show that the trends in metrics for generated CS move closer to real CS data in each of the above language pairs through the dual stage training process. We believe this viewpoint of CS as style variations opens new perspectives for modeling various tasks in CS text.