M-Eval: A Heterogeneity-Based Framework for Multi-evidence Validation in Medical RAG Systems
This work addresses reliability issues in medical question-answering systems, potentially reducing diagnostic errors, though it appears incremental as it builds on existing RAG and EBM concepts.
The paper tackles the problem of hallucinations and incorrect information in medical RAG systems by proposing M-Eval, a heterogeneity-based framework for multi-evidence validation, which improves accuracy by up to 23.31% across various LLMs.
Retrieval-augmented Generation (RAG) has demonstrated potential in enhancing medical question-answering systems through the integration of large language models (LLMs) with external medical literature. LLMs can retrieve relevant medical articles to generate more professional responses efficiently. However, current RAG applications still face problems. They generate incorrect information, such as hallucinations, and they fail to use external knowledge correctly. To solve these issues, we propose a new method named M-Eval. This method is inspired by the heterogeneity analysis approach used in Evidence-Based Medicine (EBM). Our approach can check for factual errors in RAG responses using evidence from multiple sources. First, we extract additional medical literature from external knowledge bases. Then, we retrieve the evidence documents generated by the RAG system. We use heterogeneity analysis to check whether the evidence supports different viewpoints in the response. In addition to verifying the accuracy of the response, we also assess the reliability of the evidence provided by the RAG system. Our method shows an improvement of up to 23.31% accuracy across various LLMs. This work can help detect errors in current RAG-based medical systems. It also makes the applications of LLMs more reliable and reduces diagnostic errors.