Talk Before You Retrieve: Agent-Led Discussions for Better RAG in Medical QA
This addresses the challenge of noisy and irrelevant retrievals in medical RAG systems for medical QA, offering a domain-specific enhancement.
The paper tackles the problem of hallucinations and outdated knowledge in medical question answering by proposing Discuss-RAG, a plug-and-play module that uses collaborative agent-based reasoning to improve retrieval relevance, resulting in accuracy improvements of up to 16.67% on BioASQ and 12.20% on PubMedQA.
Medical question answering (QA) is a reasoning-intensive task that remains challenging for large language models (LLMs) due to hallucinations and outdated domain knowledge. Retrieval-Augmented Generation (RAG) provides a promising post-training solution by leveraging external knowledge. However, existing medical RAG systems suffer from two key limitations: (1) a lack of modeling for human-like reasoning behaviors during information retrieval, and (2) reliance on suboptimal medical corpora, which often results in the retrieval of irrelevant or noisy snippets. To overcome these challenges, we propose Discuss-RAG, a plug-and-play module designed to enhance the medical QA RAG system through collaborative agent-based reasoning. Our method introduces a summarizer agent that orchestrates a team of medical experts to emulate multi-turn brainstorming, thereby improving the relevance of retrieved content. Additionally, a decision-making agent evaluates the retrieved snippets before their final integration. Experimental results on four benchmark medical QA datasets show that Discuss-RAG consistently outperforms MedRAG, especially significantly improving answer accuracy by up to 16.67% on BioASQ and 12.20% on PubMedQA. The code is available at: https://github.com/LLM-VLM-GSL/Discuss-RAG.