Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval
This work addresses the problem of improving the trustworthiness of agentic AI systems for developers and users by enabling fact-checking without reliance on external retrieval, which can be constrained by retrieval errors and data availability.
This paper tackles the problem of fact-checking without external retrieval, focusing on verifying arbitrary natural language claims using only the LLM's parametric knowledge. The authors introduce INTRA, a method that leverages internal model representations and achieves state-of-the-art performance across 9 datasets and 18 methods, demonstrating strong generalization.
Trustworthiness is a core research challenge for agentic AI systems built on Large Language Models (LLMs). To enhance trust, natural language claims from diverse sources, including human-written text, web content, and model outputs, are commonly checked for factuality by retrieving external knowledge and using an LLM to verify the faithfulness of claims to the retrieved evidence. As a result, such methods are constrained by retrieval errors and external data availability, while leaving the models intrinsic fact-verification capabilities largely unused. We propose the task of fact-checking without retrieval, focusing on the verification of arbitrary natural language claims, independent of their source. To study this setting, we introduce a comprehensive evaluation framework focused on generalization, testing robustness to (i) long-tail knowledge, (ii) variation in claim sources, (iii) multilinguality, and (iv) long-form generation. Across 9 datasets, 18 methods and 3 models, our experiments indicate that logit-based approaches often underperform compared to those that leverage internal model representations. Building on this finding, we introduce INTRA, a method that exploits interactions between internal representations and achieves state-of-the-art performance with strong generalization. More broadly, our work establishes fact-checking without retrieval as a promising research direction that can complement retrieval-based frameworks, improve scalability, and enable the use of such systems as reward signals during training or as components integrated into the generation process.