LLM-based Corroborating and Refuting Evidence Retrieval for Scientific Claim Verification
This addresses the challenge of verifying scientific claims using LLMs, offering an unsupervised method that generalizes across domains, though it appears incremental as an extension of existing RAG frameworks.
The paper tackles the problem of scientific claim verification by introducing CIBER, an extension of the Retrieval-Augmented Generation framework that identifies corroborating and refuting evidence documents, and it shows superior performance compared to conventional RAG approaches in evaluations with LLMs of varying linguistic proficiency.
In this paper, we introduce CIBER (Claim Investigation Based on Evidence Retrieval), an extension of the Retrieval-Augmented Generation (RAG) framework designed to identify corroborating and refuting documents as evidence for scientific claim verification. CIBER addresses the inherent uncertainty in Large Language Models (LLMs) by evaluating response consistency across diverse interrogation probes. By focusing on the behavioral analysis of LLMs without requiring access to their internal information, CIBER is applicable to both white-box and black-box models. Furthermore, CIBER operates in an unsupervised manner, enabling easy generalization across various scientific domains. Comprehensive evaluations conducted using LLMs with varying levels of linguistic proficiency reveal CIBER's superior performance compared to conventional RAG approaches. These findings not only highlight the effectiveness of CIBER but also provide valuable insights for future advancements in LLM-based scientific claim verification.