Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework
This addresses the challenge of content hallucination in LLMs, which is a critical issue for reliable text generation applications, though it appears to be an incremental improvement over existing verification methods.
The paper tackles the problem of detecting hallucinations in large language models by proposing a Markov Chain-based multi-agent debate framework that integrates claim detection, evidence retrieval, and verification, achieving significant improvements over baselines across three generative tasks.
The advent of large language models (LLMs) has facilitated the development of natural language text generation. It also poses unprecedented challenges, with content hallucination emerging as a significant concern. Existing solutions often involve expensive and complex interventions during the training process. Moreover, some approaches emphasize problem disassembly while neglecting the crucial validation process, leading to performance degradation or limited applications. To overcome these limitations, we propose a Markov Chain-based multi-agent debate verification framework to enhance hallucination detection accuracy in concise claims. Our method integrates the fact-checking process, including claim detection, evidence retrieval, and multi-agent verification. In the verification stage, we deploy multiple agents through flexible Markov Chain-based debates to validate individual claims, ensuring meticulous verification outcomes. Experimental results across three generative tasks demonstrate that our approach achieves significant improvements over baselines.