Osiris: A Lightweight Open-Source Hallucination Detection System
This addresses the scaling issues of hallucination detection for RAG system builders by providing a cost-effective and faster alternative to human evaluation or closed-source models.
The paper tackles the problem of hallucinations in Retrieval-Augmented Generation (RAG) systems by introducing a lightweight open-source detection system, achieving better recall than GPT-4o on a benchmark with a 7B model while maintaining competitive precision and accuracy.
Retrieval-Augmented Generation (RAG) systems have gained widespread adoption by application builders because they leverage sources of truth to enable Large Language Models (LLMs) to generate more factually sound responses. However, hallucinations, instances of LLM responses that are unfaithful to the provided context, often prevent these systems from being deployed in production environments. Current hallucination detection methods typically involve human evaluation or the use of closed-source models to review RAG system outputs for hallucinations. Both human evaluators and closed-source models suffer from scaling issues due to their high costs and slow inference speeds. In this work, we introduce a perturbed multi-hop QA dataset with induced hallucinations. Via supervised fine-tuning on our dataset, we achieve better recall with a 7B model than GPT-4o on the RAGTruth hallucination detection benchmark and offer competitive performance on precision and accuracy, all while using a fraction of the parameters. Code is released at our repository.