MedChat: A Multi-Agent Framework for Multimodal Diagnosis with Large Language Models
This work addresses the problem of improving clinical reporting efficiency and mitigating ophthalmologist shortages in healthcare, though it appears incremental by building on existing LLM-based approaches.
The authors tackled the challenge of applying large language models (LLMs) to medical imaging by proposing MedChat, a multi-agent framework that combines specialized vision models with role-specific LLM agents, resulting in enhanced reliability and reduced hallucination risk for automated glaucoma diagnosis.
The integration of deep learning-based glaucoma detection with large language models (LLMs) presents an automated strategy to mitigate ophthalmologist shortages and improve clinical reporting efficiency. However, applying general LLMs to medical imaging remains challenging due to hallucinations, limited interpretability, and insufficient domain-specific medical knowledge, which can potentially reduce clinical accuracy. Although recent approaches combining imaging models with LLM reasoning have improved reporting, they typically rely on a single generalist agent, restricting their capacity to emulate the diverse and complex reasoning found in multidisciplinary medical teams. To address these limitations, we propose MedChat, a multi-agent diagnostic framework and platform that combines specialized vision models with multiple role-specific LLM agents, all coordinated by a director agent. This design enhances reliability, reduces hallucination risk, and enables interactive diagnostic reporting through an interface tailored for clinical review and educational use. Code available at https://github.com/Purdue-M2/MedChat.