A Multi-Agent System for Complex Reasoning in Radiology Visual Question Answering
This work addresses the need for reliable and interpretable AI in clinical settings, specifically for radiologists, though it appears incremental as it builds on existing multi-agent and multimodal methods.
The paper tackled the problem of factual accuracy, hallucinations, and cross-modal misalignment in radiology visual question answering (RVQA) by introducing a multi-agent system, which outperformed strong multimodal large language model baselines on a challenging dataset curated via model disagreement filtering.
Radiology visual question answering (RVQA) provides precise answers to questions about chest X-ray images, alleviating radiologists' workload. While recent methods based on multimodal large language models (MLLMs) and retrieval-augmented generation (RAG) have shown promising progress in RVQA, they still face challenges in factual accuracy, hallucinations, and cross-modal misalignment. We introduce a multi-agent system (MAS) designed to support complex reasoning in RVQA, with specialized agents for context understanding, multimodal reasoning, and answer validation. We evaluate our system on a challenging RVQA set curated via model disagreement filtering, comprising consistently hard cases across multiple MLLMs. Extensive experiments demonstrate the superiority and effectiveness of our system over strong MLLM baselines, with a case study illustrating its reliability and interpretability. This work highlights the potential of multi-agent approaches to support explainable and trustworthy clinical AI applications that require complex reasoning.