Analysing Cross-Lingual Transfer in Low-Resourced African Named Entity Recognition
This work addresses the challenge of improving NLP for low-resourced languages, though it is incremental as it builds on existing transfer learning methods.
The study investigated cross-lingual transfer learning for named entity recognition in ten low-resourced African languages, finding that models excelling in single languages often generalize poorly to others, and data overlap between source and target datasets better predicts transfer performance than linguistic or geographic distance.
Transfer learning has led to large gains in performance for nearly all NLP tasks while making downstream models easier and faster to train. This has also been extended to low-resourced languages, with some success. We investigate the properties of cross-lingual transfer learning between ten low-resourced languages, from the perspective of a named entity recognition task. We specifically investigate how much adaptive fine-tuning and the choice of transfer language affect zero-shot transfer performance. We find that models that perform well on a single language often do so at the expense of generalising to others, while models with the best generalisation to other languages suffer in individual language performance. Furthermore, the amount of data overlap between the source and target datasets is a better predictor of transfer performance than either the geographical or genetic distance between the languages.