Automatic Correction of Writing Anomalies in Hausa Texts
This work addresses text quality issues for Hausa NLP applications, with incremental contributions through dataset creation and model adaptation for a low-resource language.
The paper tackled the problem of writing anomalies in Hausa texts, such as character and spacing errors, by finetuning transformer-based models on a synthetic parallel dataset of over 450,000 sentence pairs, resulting in significant improvements in F1, BLEU, METEOR scores and reductions in CER and WER.
Hausa texts are often characterized by writing anomalies such as incorrect character substitutions and spacing errors, which sometimes hinder natural language processing (NLP) applications. This paper presents an approach to automatically correct the anomalies by finetuning transformer-based models. Using a corpus gathered from several public sources, we created a large-scale parallel dataset of over 450,000 noisy-clean Hausa sentence pairs by introducing synthetically generated noise, fine-tuned to mimic realistic writing errors. Moreover, we adapted several multilingual and African language-focused models, including M2M100, AfriTEVA, mBART, and Opus-MT variants for this correction task using SentencePiece tokenization. Our experimental results demonstrate significant increases in F1, BLEU and METEOR scores, as well as reductions in Character Error Rate (CER) and Word Error Rate (WER). This research provides a robust methodology, a publicly available dataset, and effective models to improve Hausa text quality, thereby advancing NLP capabilities for the language and offering transferable insights for other low-resource languages.