Translation Aligned Sentence Embeddings for Turkish Language
This addresses a domain-specific need for Turkish language processing, but the method is incremental as it adapts existing techniques to a new language context.
The authors tackled the problem of limited high-quality datasets for training sentence embeddings in Turkish by proposing a two-stage fine-tuning method for a pretrained encoder-decoder model, achieving high accuracy with short training times on limited data.
Due to the limited availability of high quality datasets for training sentence embeddings in Turkish, we propose a training methodology and a regimen to develop a sentence embedding model. The central idea is simple but effective : is to fine-tune a pretrained encoder-decoder model in two consecutive stages, where the first stage involves aligning the embedding space with translation pairs. Thanks to this alignment, the prowess of the main model can be better projected onto the target language in a sentence embedding setting where it can be fine-tuned with high accuracy in short duration with limited target language dataset.