Combining Distantly Supervised Models with In Context Learning for Monolingual and Cross-Lingual Relation Extraction
This work addresses noisy annotation issues in DSRE for NLP researchers and practitioners, extending to cross-lingual settings for low-resource languages, though it is incremental as it builds on existing DSRE and ICL methods.
The paper tackles the challenge of distantly supervised relation extraction (DSRE) by proposing HYDRE, a hybrid framework that combines a trained DSRE model with in-context learning using large language models, achieving up to 20 F1 point gains in English and an average of 17 F1 points on Indic languages over prior state-of-the-art models.
Distantly Supervised Relation Extraction (DSRE) remains a long-standing challenge in NLP, where models must learn from noisy bag-level annotations while making sentence-level predictions. While existing state-of-the-art (SoTA) DSRE models rely on task-specific training, their integration with in-context learning (ICL) using large language models (LLMs) remains underexplored. A key challenge is that the LLM may not learn relation semantics correctly, due to noisy annotation. In response, we propose HYDRE -- HYbrid Distantly Supervised Relation Extraction framework. It first uses a trained DSRE model to identify the top-k candidate relations for a given test sentence, then uses a novel dynamic exemplar retrieval strategy that extracts reliable, sentence-level exemplars from training data, which are then provided in LLM prompt for outputting the final relation(s). We further extend HYDRE to cross-lingual settings for RE in low-resource languages. Using available English DSRE training data, we evaluate all methods on English as well as a newly curated benchmark covering four diverse low-resource Indic languages -- Oriya, Santali, Manipuri, and Tulu. HYDRE achieves up to 20 F1 point gains in English and, on average, 17 F1 points on Indic languages over prior SoTA DSRE models. Detailed ablations exhibit HYDRE's efficacy compared to other prompting strategies.