Very Low Resource Sentence Alignment: Luhya and Swahili
This work addresses the challenge of creating parallel corpora for low-resource African languages, which is incremental as it applies existing methods to new data with fine-tuning.
The researchers tackled the problem of sentence alignment for low-resource languages like Luhya and Swahili using pre-trained models, finding that LaBSE outperformed LASER but both performed poorly on Luhya initially, with fine-tuning improving LaBSE's alignment accuracy from 22.0% to 53.3%.
Language-agnostic sentence embeddings generated by pre-trained models such as LASER and LaBSE are attractive options for mining large datasets to produce parallel corpora for low-resource machine translation. We test LASER and LaBSE in extracting bitext for two related low-resource African languages: Luhya and Swahili. For this work, we created a new parallel set of nearly 8000 Luhya-English sentences which allows a new zero-shot test of LASER and LaBSE. We find that LaBSE significantly outperforms LASER on both languages. Both LASER and LaBSE however perform poorly at zero-shot alignment on Luhya, achieving just 1.5% and 22.0% successful alignments respectively (P@1 score). We fine-tune the embeddings on a small set of parallel Luhya sentences and show significant gains, improving the LaBSE alignment accuracy to 53.3%. Further, restricting the dataset to sentence embedding pairs with cosine similarity above 0.7 yielded alignments with over 85% accuracy.