Mafoko: Structuring and Building Open Multilingual Terminologies for South African NLP
This addresses the problem of fragmented language resources for South African NLP, enabling more equitable multilingual technologies, though it is incremental as it builds on existing terminology lists.
The paper tackled the lack of structured terminological data for South African languages by creating the Mafoko dataset, which improved English-to-Tshivenda machine translation accuracy and consistency in large language models through integration into a RAG pipeline.
The critical lack of structured terminological data for South Africa's official languages hampers progress in multilingual NLP, despite the existence of numerous government and academic terminology lists. These valuable assets remain fragmented and locked in non-machine-readable formats, rendering them unusable for computational research and development. Mafoko addresses this challenge by systematically aggregating, cleaning, and standardising these scattered resources into open, interoperable datasets. We introduce the foundational Mafoko dataset, released under the equitable, Africa-centered NOODL framework. To demonstrate its immediate utility, we integrate the terminology into a Retrieval-Augmented Generation (RAG) pipeline. Experiments show substantial improvements in the accuracy and domain-specific consistency of English-to-Tshivenda machine translation for large language models. Mafoko provides a scalable foundation for developing robust and equitable NLP technologies, ensuring South Africa's rich linguistic diversity is represented in the digital age.