Disentangling Codemixing in Chats: The NUS ABC Codemixed Corpus
This provides a foundational dataset for researchers in computational linguistics and NLP to study natural multilingual communication patterns, though it is incremental as it focuses on data collection rather than new methods.
The study tackled the lack of publicly available, labeled corpora for code-mixing in multilingual chats by introducing the NUS ABC Codemixed Corpus, which includes over 355,641 messages with detailed metadata and linguistic statistics.
Code-mixing involves the seamless integration of linguistic elements from multiple languages within a single discourse, reflecting natural multilingual communication patterns. Despite its prominence in informal interactions such as social media, chat messages and instant-messaging exchanges, there has been a lack of publicly available corpora that are author-labeled and suitable for modeling human conversations and relationships. This study introduces the first labeled and general-purpose corpus for understanding code-mixing in context while maintaining rigorous privacy and ethical standards. Our live project will continuously gather, verify, and integrate code-mixed messages into a structured dataset released in JSON format, accompanied by detailed metadata and linguistic statistics. To date, it includes over 355,641 messages spanning various code-mixing patterns, with a primary focus on English, Mandarin, and other languages. We expect the Codemix Corpus to serve as a foundational dataset for research in computational linguistics, sociolinguistics, and NLP applications.