EvidenceRL: Reinforcing Evidence Consistency for Trustworthy Language Models
This addresses the critical issue of untrustworthy outputs in high-stakes domains like healthcare and law, where decisions must be evidence-based, representing a strong specific gain rather than a foundational breakthrough.
The paper tackles the problem of hallucinations in large language models by introducing EvidenceRL, a reinforcement learning framework that enforces evidence adherence during training, resulting in significant improvements in evidence grounding and faithfulness across cardiac diagnosis and legal reasoning domains, with metrics like F1@3 increasing from 37.0 to 54.5 and hallucinations dropping nearly 5×.
Large Language Models (LLMs) are fluent but prone to hallucinations, producing answers that appear plausible yet are unsupported by available evidence. This failure is especially problematic in high-stakes domains where decisions must be justified by verifiable information. We introduce \textbf{EvidenceRL}, a reinforcement learning framework that enforces evidence adherence during training. EvidenceRL scores candidate responses for grounding (entailment with retrieved evidence and context) and correctness (agreement with reference answers) and optimizes the generator using Group Relative Policy Optimization (GRPO). We evaluate across two high-stakes domains, cardiac diagnosis and legal reasoning, where EvidenceRL consistently improves evidence grounding and faithfulness without sacrificing task accuracy. On cardiac diagnosis, F1@3 increases from 37.0 to 54.5 on Llama-3.2-3B while grounding ($G_{\max}@3$) rises from 47.6 to 78.2; hallucinations drop nearly 5$\times$ and evidence-supported diagnoses increase from 31.8\% to 61.6\%. On legal reasoning, EvidenceRL raises Faithfulness from 32.8\% to 67.6\% on Llama-3.1-8B, demonstrating consistent behavioral change across domains. Our code is open-sourced at https://github.com/Wizaaard/EvidenceRL.git.