Translation and Fusion Improves Zero-shot Cross-lingual Information Extraction
This work addresses the performance gap in information extraction for low-resource languages, offering a practical solution with incremental improvements over existing methods.
The paper tackles the problem of limited applicability of large language models to low-resource languages in information extraction by proposing TransFusion, a framework that uses English translations and annotation fusion, resulting in improved zero-shot cross-lingual transfer and significant F1 score gains (e.g., +5 to +14 F1) across various models and datasets.
Large language models (LLMs) combined with instruction tuning have shown significant progress in information extraction (IE) tasks, exhibiting strong generalization capabilities to unseen datasets by following annotation guidelines. However, their applicability to low-resource languages remains limited due to lack of both labeled data for fine-tuning, and unlabeled text for pre-training. In this paper, we propose TransFusion, a framework in which models are fine-tuned to use English translations of low-resource language data, enabling more precise predictions through annotation fusion. Based on TransFusion, we introduce GoLLIE-TF, a cross-lingual instruction-tuned LLM for IE tasks, designed to close the performance gap between high and low-resource languages. Our experiments across twelve multilingual IE datasets spanning 50 languages demonstrate that GoLLIE-TF achieves better zero-shot cross-lingual transfer over the base model. In addition, we show that TransFusion significantly improves low-resource language named entity recognition when applied to proprietary models such as GPT-4 (+5 F1) with a prompting approach, or fine-tuning different language models including decoder-only (+14 F1) and encoder-only (+13 F1) architectures.