R1-RE: Cross-Domain Relation Extraction with RLVR
This addresses robustness issues in relation extraction for NLP applications, though it appears incremental as it adapts existing RL concepts to a specific domain.
The paper tackles poor out-of-domain generalization in relation extraction by reframing it as a reasoning task with a reinforcement learning framework (RLVR), achieving approximately 70% OOD accuracy on datasets like Sem-2010 and MDKG, matching proprietary models like GPT-4o.
Relation extraction (RE) is a core task in natural language processing. Traditional approaches typically frame RE as a supervised learning problem, directly mapping context to labels-an approach that often suffers from poor out-of-domain (OOD) generalization. Inspired by the workflow of human annotators, we reframe RE as a reasoning task guided by annotation guidelines and introduce R1-RE, the first reinforcement learning with verifiable reward (RLVR) framework for RE tasks. Our method elicits the reasoning abilities of small language models for annotation tasks, resulting in significantly improved OOD robustness. We evaluate our approach on the public Sem-2010 dataset and a private MDKG dataset. The R1-RE-7B model attains an average OOD accuracy of approximately 70%, on par with leading proprietary models such as GPT-4o. Additionally, our comprehensive analysis provides novel insights into the training dynamics and emergent reasoning behaviors of the RLVR paradigm for RE.