\textsc{CantoNLU}: A benchmark for Cantonese natural language understanding
This addresses the problem of under-resourced Cantonese NLP for researchers and developers, but it is incremental as it primarily provides a new benchmark and baseline evaluations.
The authors tackled the scarcity of evaluation frameworks for Cantonese by introducing CantoNLU, a benchmark covering seven NLU tasks, and found that Cantonese-adapted models performed best overall, with monolingual models excelling in syntactic tasks and Mandarin models remaining competitive in some settings.
Cantonese, although spoken by millions, remains under-resourced due to policy and diglossia. To address this scarcity of evaluation frameworks for Cantonese, we introduce \textsc{\textbf{CantoNLU}}, a benchmark for Cantonese natural language understanding (NLU). This novel benchmark spans seven tasks covering syntax and semantics, including word sense disambiguation, linguistic acceptability judgment, language detection, natural language inference, sentiment analysis, part-of-speech tagging, and dependency parsing. In addition to the benchmark, we provide model baseline performance across a set of models: a Mandarin model without Cantonese training, two Cantonese-adapted models obtained by continual pre-training a Mandarin model on Cantonese text, and a monolingual Cantonese model trained from scratch. Results show that Cantonese-adapted models perform best overall, while monolingual models perform better on syntactic tasks. Mandarin models remain competitive in certain settings, indicating that direct transfer may be sufficient when Cantonese domain data is scarce. We release all datasets, code, and model weights to facilitate future research in Cantonese NLP.