Adapting to the Low-Resource Double-Bind: Investigating Low-Compute Methods on Low-Resource African Languages
This addresses the barrier to NLP research for low-resource African languages, offering a cost-effective solution, though it is incremental as it applies existing adapter methods to new data.
The paper tackled the problem of limited computational resources and data scarcity for African languages in NLP by exploring low-compute methods like language adapters, finding that these adapters achieve performance comparable to massive pre-trained models using only free compute resources.
Many natural language processing (NLP) tasks make use of massively pre-trained language models, which are computationally expensive. However, access to high computational resources added to the issue of data scarcity of African languages constitutes a real barrier to research experiments on these languages. In this work, we explore the applicability of low-compute approaches such as language adapters in the context of this low-resource double-bind. We intend to answer the following question: do language adapters allow those who are doubly bound by data and compute to practically build useful models? Through fine-tuning experiments on African languages, we evaluate their effectiveness as cost-effective approaches to low-resource African NLP. Using solely free compute resources, our results show that language adapters achieve comparable performances to massive pre-trained language models which are heavy on computational resources. This opens the door to further experimentation and exploration on full-extent of language adapters capacities.