Code-Switching Language Modeling using Syntax-Aware Multi-Task Learning
This work addresses the challenge of language modeling for code-switched speech in low-resource settings, which is incremental as it builds on existing methods with specific enhancements.
The paper tackled the problem of code-switching language modeling by introducing a multi-task learning model that shares syntax representations to leverage linguistic information and address low-resource data issues, resulting in improvements of 9.7% and 7.4% in perplexity on SEAME datasets.
Lack of text data has been the major issue on code-switching language modeling. In this paper, we introduce multi-task learning based language model which shares syntax representation of languages to leverage linguistic information and tackle the low resource data issue. Our model jointly learns both language modeling and Part-of-Speech tagging on code-switched utterances. In this way, the model is able to identify the location of code-switching points and improves the prediction of next word. Our approach outperforms standard LSTM based language model, with an improvement of 9.7% and 7.4% in perplexity on SEAME Phase I and Phase II dataset respectively.