Are Multilingual Models Effective in Code-Switching?
This addresses the problem of efficient and practical natural language processing for code-switching scenarios, but it is incremental as it builds on existing methods.
The paper investigated whether multilingual language models are effective for code-switching tasks, finding that they do not guarantee high-quality representations, while meta-embeddings achieve similar results with significantly fewer parameters.
Multilingual language models have shown decent performance in multilingual and cross-lingual natural language understanding tasks. However, the power of these multilingual models in code-switching tasks has not been fully explored. In this paper, we study the effectiveness of multilingual language models to understand their capability and adaptability to the mixed-language setting by considering the inference speed, performance, and number of parameters to measure their practicality. We conduct experiments in three language pairs on named entity recognition and part-of-speech tagging and compare them with existing methods, such as using bilingual embeddings and multilingual meta-embeddings. Our findings suggest that pre-trained multilingual models do not necessarily guarantee high-quality representations on code-switching, while using meta-embeddings achieves similar results with significantly fewer parameters.